[DOCS] Adds Input Validation / Type Checking doc for custom expectations #4092
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
✔️ Deploy Preview for niobium-lead-7998 ready! 🔨 Explore the source changes: aac7952 🔍 Inspect the deploy log: https://app.netlify.com/sites/niobium-lead-7998/deploys/61fab84d87cdcb00074324fb 😎 Browse the preview: https://deploy-preview-4092--niobium-lead-7998.netlify.app |
austiezr
requested review from
Rachel-Reverie
and removed request for
donaldheppner
HOWDY! This is your friendly 🤖 CHANGELOG bot 🤖Please don't forget to add a clear and succinct description of your change under the Develop header in ✨ Thank you! ✨ |
Rachel-Reverie
suggested changes
Feb 2, 2022
.../expectations/features_custom_expectations/how_to_add_input_validation_for_an_expectation.md
Outdated
Show resolved
Hide resolved
.../expectations/features_custom_expectations/how_to_add_input_validation_for_an_expectation.md
Outdated
Show resolved
Hide resolved
.../expectations/features_custom_expectations/how_to_add_input_validation_for_an_expectation.md
Outdated
Show resolved
Hide resolved
.../expectations/features_custom_expectations/how_to_add_input_validation_for_an_expectation.md
Outdated
Show resolved
Hide resolved
…dd_input_validation_for_an_expectation.md Co-authored-by: Rachel-Reverie <[email protected]>
…com/great-expectations/great_expectations into docs/input_validation_type_checking
kenwade4
added a commit
that referenced
this pull request
Feb 23, 2022
* Create how_to_enable_additional_execution_engines_for_custom_expectations.md add documentation for implementing metrics in SQLAlchemy and Spark * initial commit of custom expectations docs restructure * clean up structure and titles * put creating expectation doc scripts under test * - Reworks custom expectation doc into a more explicit guide for custom column aggregate expectations - Fixes pointers in other docs to this doc - Brings supporting example into great_expectations from superconductive/ge_tutorials * cleanup on column_aggregate docs | skeleton for column_map doc * fixed references to map_metric * moving reference file under tests, fixing references * Rename custom_column_max_example.py * Restore colors to `contributing maturity.md` (#3910) * Draft overview for custom_expectations (#3911) * Draft Overview for custom_expectations * Add creating_custom_expectations/components/prerequisites.jsx * Feature/expectation completeness checklist (#3906) * Implement the first several steps of generate_diagnostic_checklist, with tests and unit-testable supporting methods. * Refactor tests so that test_run_diagnostics uses the same test cases as test_generate_diagnostic_checklist * Move expect_column_values_to_equal_three.py to a fixtures directory * expansion of contributing_maturity with specifics for expectations * Updates and edits to custom expectations overview and contributing maturity docs * progress on adapting column agg doc to fit pattern from column map doc | edits to template and example to support | linting * rework of agg doc re: map doc | updates to supporting code | linting and typo fixes * updates to agg doc | small edits to metrics & prereqs * Feature/refactor and type expectation diagnostic report (#3929) * Initial scaffolding. One type and one test * Scaffold most of the rest of the objects we'll need, with some tests. * Add tests to unpack the behavior of @DataClass combined with SerializableDotDict * More WIP, especially tests * A spate of renaming and tightening * Add SerializableDotDict.keys * Update expectation_test_data_cases to use SerializableDictDot * Fix gtypos * Update SerializableDictDot, with tests * The meat of the refactor, mostly done with some WIP * Fixed test__count_positive_and_negative_example_cases * Decruft * Fix test_include_in_gallery_flag * Make .to_dict recursive * Linting * Remove pydantic; Remove submessages in execution_engines; test_generate_diagnostic_checklist__first_iteration now passes * Refactor _count_unexpected_cases_and_get_sub_messages into _count_unexpected_cases * Fix docstring * Clean up types and methods around LibraryMetadata * Add support for enum serialization * Implement LegacyAugmentedLibraryMetadataAdapter; Tidy up a little * Minor bugfixes. Many docstrings * Add docstrings for test_data_cases * Improve _get_metric_diagnostics_list * Clean up _get_test_results * Work on docstrings and more bug fixes. Re-up pydantic. Need to get out of the weeds soon. * Deprecate LegacyAugmentedLibraryMetadataAdapter * Tidy up rendering methods * Add RendererTestDiagnostics object * Add unit tests for _get_augmented_library_metadata and _get_examples * Unit test for _get_description_diagnostics * Implement OtherExpectationTestDiagnostics * Flow typed ExecutedExpectationTestCases through * Remove dependence on global _registered_renderers. Get another test to pass. * Clean up tests and get business logic for Has at least one positive and negative example case, and all test cases pass working * Decruft. Skip one failing test * Feature/changes from novuum jan3 review (#3941) * Add ExpectationDiagnosticMaturityMessages class and refactor around it * Add optional AugmentedLibraryMetadata.package and ExpectationDiagnosticCheckMessage.doc_url * Decruft * Fix mismatched string in tests * Delete test_expectation__get_execution_engine_dict _get_execution_engine_dict was replaced by _get_execution_engine_diagnostics * Update tests/expectations/test_expectation_diagnostics.py Co-authored-by: Don Heppner <[email protected]> * Implement ExpectationDiagnostics.to_json_dict method and use in build_gallery.py script * Update build_gallery.py script to collect expectation tracebacks and display before writing gallery json file * Allow ExpectationTestDataCases to receive 'schemas' * Add commented-out 'test_backends' attribute to ExpectationTestDataCases class * Add a mysql docker image that works with M1 Macs % docker-compose up Pulling mysql_db (mysql:8.0.20)... 8.0.20: Pulling from library/mysql ERROR: no matching manifest for linux/arm64/v8 in the manifest list entries * Ensure that the 'configuration' object passed to renderers are typed objects * Update build_gallery.py script to not die if there are expectation tracebacks * Use dot notation for args passed to ExpectationDiagnosticCheckMessage Co-authored-by: Ken Wade <[email protected]> Co-authored-by: Don Heppner <[email protected]> * [FEATURE] Add maturity_checklist to ExpectationDiagnostics (#4061) * Add maturity_checklist to ExpectationDiagnostics and use in Expectation.run_diagnostics * Remove concept_only from ExpectationDiagnosticMaturityMessages * Re-format files with black * Update the 'broken links' identified by docs_link_checker.py script * [DOCS] WIP Docs/custom expectation test cases (#3969) * updated opening paragraph * further edits * completed ex_cases doc | re-orged | edited sidebar & example file to support * fixed codeblock line refs * Update docs/guides/expectations/features_custom_expectations/how_to_add_example_cases_for_an_expectation.md Co-authored-by: Rachel-Reverie <[email protected]> * Update docs/guides/expectations/features_custom_expectations/how_to_add_example_cases_for_an_expectation.md Co-authored-by: Rachel-Reverie <[email protected]> * Update docs/guides/expectations/features_custom_expectations/how_to_add_example_cases_for_an_expectation.md Co-authored-by: Rachel-Reverie <[email protected]> * Apply suggestions from code review Co-authored-by: Rachel-Reverie <[email protected]> Co-authored-by: Rachel-Reverie <[email protected]> * removed errant print | removed maturity tag * removed errant print | removed maturity tag * fixed maturity verbiage * fixed code refs | fixed diagnostics verbiage | expanded contribution stub * added details on testing for errors * fix to diagnostics verbiage * [BUGFIX] Pin `pip --upgrade` to a specific version to bypass bug in pip's underlying algorithm (#4100) * fix: pin pip upgrade * chore: test change to trigger pipeline * fix: also pin dependency graph pip upgrades * chore: revert changes * [BUGFIX] Use `pip==20.2.4` for usage statistics stage of CI/CD (#4102) * fix: pin pip upgrade * chore: test change to trigger pipeline * fix: also pin dependency graph pip upgrades * chore: revert changes * chore: revert usage stats pip versions * [MAINTENANCE] Fix Configuration Schema and Refactor Rule-Based Profiler; Initial Implementation of Reconciliation Logic Between Configuration and Runtime Arguments (#4088) * [BUGFIX] Fix shared state issue in renderer test (#4000) * chore: add assert to test problem * test: use abspath to find JSON files * test: add UNSUPPORTED_EXPECTATIONS * chore: add comment to explain test * refactor: remove defaultdict * chore: update comment * fix: bring back defaultdict * chore: fix typo * chore: update comments * fix: use context manager pattern for cd * chore: move certain asserts out of context manager scope * refactor: use context manager cd for tests * chore: remove unnecessary imports * test: fix test identation * refactor: split up os join usage * bringing example code for agg doc under test * removing as a required key for library_metadata * replaced deprecated alias * replaced deprecated alias * [FEATURE] Add classes for Great Expectations contrib packaging JSON object (#3922) * feat: add JSON object classes * feat: add script to JSON parse * chore: update enum value per @abegong * refactor: move types into CLI tool * feat: add diagnostic parsing support to package class * feat: implement dynamic importing to retrieve Expectations * refactor: use issubclass checks * feat: misc cleanup to ensure proper serialization * fix: misc fixes to get hook working * chore: update logger calls * chore: add example Package JSON * refactor: use enum for social media types * feat: make dataclasses frozen * chore: make to_json_file func private * chore: remove unnecessary imports * refactor: rename types to package * chore: add better exception/errors * chore: misc comments * refactor: move serialization methods out of class * refactor: use Config suffix * refactor: remove Config in place of Manifest * fix: rename config to manifest * [DOCS] Adds Input Validation / Type Checking doc for custom expectations (#4092) * implement validate_configuraton method * typo fixes * typo fixes * fleshed out input_validation doc * Update docs/guides/expectations/features_custom_expectations/how_to_add_input_validation_for_an_expectation.md Co-authored-by: Rachel-Reverie <[email protected]> * edits for content and clarity * Update how_to_add_input_validation_for_an_expectation.md Co-authored-by: Rachel-Reverie <[email protected]> * WIP Docs/how to create custom column map expectations WIP (#3914) * * Rough draft text for How to create a custom Column map Expectation * Add How to create a Custom column Map Expectation to sidebar.js * Simplify the template for column map expectations. * Begin to implement generate_diagnostic_checklist, with tests. WIP * Drop extraneous print statement in self_check/util.py. * Refactor generate_diagnostic_checklist to return a string; Get tests working * Factor out _convert_checks_into_output_message and add tests * Implement logic for Check whether core logic for this Expectation exists and passes tests on at least one Execution Engine * Refactor tests so that test_run_diagnostics uses the same test cases as test_generate_diagnostic_checklist * Factor out and test _count_positive_and_negative_example_cases * Add a good docstring test * Factor out _count_unexpected_cases_and_get_sub_messages * Revert sidebars.js * Revert three files that shouldn't have been on this branch to begin with * Update how_to_create_custom_column_map_expectations.md with draft text. Update corresponding Expectation template. * black * isort * Update changelog * Improve template * Improve code style guide * Full text for the first two sections. Make step numbers sane. * Add instructions for example cases. * Drop tests/expectations/expect_column_values_to_equal_three.py * * Clean up create_custom_expectations/components/prerequisites.jsx * Still writing and writing * finish draft text, with corresponding changes to the template file * First friction log * Tidy up Other Parameters * Add integration test fixture * Fixes codeblock references in guide | Brings template & example file in line with previously in-line codeblocks | Minor formatting, linting, typo fixes * unifying completeness checklist blocks * typo fixes * fixing broken links * linting * edits to map doc & prereqs * added block addressing _validate * edits to template/example | fixes to code refs | expansions to testing & contribution | fixes to diagnostics verbiage Co-authored-by: Austin Robinson <[email protected]> Co-authored-by: Austin Ziech Robinson <[email protected]> Co-authored-by: Ken Wade <[email protected]> * bringing map example code under test * fixed code refs * fixed code refs * fixed code refs * [DOCS] Adds docs for implementing Spark & SQLAlchemy backends for Custom Expectations (#4162) * updated example scripts with spark/sqlalchemy backends * Complete SQLAlchemy Doc * minor verbiage changes * Complete Spark Doc * fixed code refs * [FEATURE] Add parsing of Expectation diagnostics to contrib packaging JSON object (#4114) * feat: add JSON object classes * feat: add script to JSON parse * chore: update enum value per @abegong * refactor: move types into CLI tool * feat: add diagnostic parsing support to package class * feat: implement dynamic importing to retrieve Expectations * refactor: use issubclass checks * feat: misc cleanup to ensure proper serialization * fix: misc fixes to get hook working * chore: update logger calls * chore: add example Package JSON * refactor: use enum for social media types * feat: make dataclasses frozen * chore: make to_json_file func private * chore: remove unnecessary imports * refactor: rename types to package * chore: add better exception/errors * chore: misc comments * refactor: move serialization methods out of class * refactor: use Config suffix * refactor: remove Config in place of Manifest * fix: rename config to manifest * feat: start parsing diagnostics * feat: parse requirements * feat: update requirements parsing * refactor: make manifest mutable * feat: continue impl * feat: continue update attr feature * test: add tests for update methods * chore: revert files back to hackathon-docs state * fix: push unlinted changes * chore: revert docs * chore: revert azure yml * chore: delete extraneous files * chore: revert additional files * chore: revert final files * test: add test for static attrs * chore: revert cli.py * feat: exit early with bad requirements path * feat: ensure enums are JSON serializable * feat: add sync cmd * chore: update comments * chore: additional cleanup * chore: misc updates * feat: add parsing support for package_info.yml * test: start writing tests for addl parsing * fix: fix filenotfound error issue * feat: write build script for CI/CD uploading to S3 * chore: misc updates * chore: misc cleanup * chore: bump black requirement * [DOCS] "How to contribute a Great Expectations package" (#3932) * docs: create how-to-guide file * docs: start guide * docs: write first version of doc * updated contributing_package & supporting files * style fixes; * docs: update doc * Update contributing_package.md Split grouped terminal commands into separate instructions Co-authored-by: Austin Robinson <[email protected]> Co-authored-by: Austin Ziech Robinson <[email protected]> Co-authored-by: Rachel-Reverie <[email protected]> * import fix * fixing broken links * fix: update import statement to ensure CI/CD script works * fix: ensure that build script picks up contrib packages * fix: ensure CI/CD script works from scripts/ and project root * [DOCS] How to add Statement Renderers to Custom Expectations (#4236) * new intro * renaming, links * initial work * implementing renderers for example scripts * statement renderer doc * typo fix * linting and code ref fixes * fixing code refs * fixing code refs * fixing code refs * Fixing code refs * fixing code refs * [DOCS] How to contribute a Custom Expectation (#4220) * reorg, fixes to pointers & sidebar * started on doc update * first pass rework * content and link fixes * added reqs * created custom expectation targeted contribution doc * linking fixes * reorg and linking fixes * Update expectations.md * commenting out unused docs from sidebar * fixes re: 02/09 rev * fixes re: 02/09 rev * fixes re: 02/09 rev * [DOCS] Updates Custom Expectations Overview (#4237) * linking and verbiage * fixes to linting doc * [DOCS] Docs/creating custom expectations/overview (#4238) * linking and verbiage * fixes to linting doc * linking * [MAINTENANCE] Refactor generate_expectation_tests and run Expectation tests as part of diagnostics (#3951) * Initial scaffolding. One type and one test * Scaffold most of the rest of the objects we'll need, with some tests. * Add tests to unpack the behavior of @DataClass combined with SerializableDotDict * More WIP, especially tests * A spate of renaming and tightening * Add SerializableDotDict.keys * Update expectation_test_data_cases to use SerializableDictDot * Fix gtypos * Update SerializableDictDot, with tests * The meat of the refactor, mostly done with some WIP * Fixed test__count_positive_and_negative_example_cases * Decruft * Fix test_include_in_gallery_flag * Make .to_dict recursive * Linting * Remove pydantic; Remove submessages in execution_engines; test_generate_diagnostic_checklist__first_iteration now passes * Refactor _count_unexpected_cases_and_get_sub_messages into _count_unexpected_cases * Fix docstring * Clean up types and methods around LibraryMetadata * Add support for enum serialization * Implement LegacyAugmentedLibraryMetadataAdapter; Tidy up a little * Small rename * Remove pydantic * Remove pydantic * Remove pydantic * Minor bugfixes. Many docstrings * Add docstrings for test_data_cases * Improve _get_metric_diagnostics_list * Clean up _get_test_results * Work on docstrings and more bug fixes. Re-up pydantic. Need to get out of the weeds soon. * Deprecate LegacyAugmentedLibraryMetadataAdapter * Tidy up rendering methods * Add RendererTestDiagnostics object * Add unit tests for _get_augmented_library_metadata * Add unit tests for _get_augmented_library_metadata and _get_examples * Unit test for _get_description_diagnostics * Implement OtherExpectationTestDiagnostics * Flow typed ExecutedExpectationTestCases through * Remove dependence on global _registered_renderers. Get another test to pass. * Get another test to pass * Fix another test * Fix another test * Clean up tests and get business logic for Has at least one positive and negative example case, and all test cases pass working * Decruft. Skip one failing test * black * isort * Comment out pydantic * isort and black * Lint manually * Lint manually * Feature/changes from novuum jan3 review (#3941) * Add ExpectationDiagnosticMaturityMessages class and refactor around it * Add optional AugmentedLibraryMetadata.package and ExpectationDiagnosticCheckMessage.doc_url * Decruft * Linting * Fix mismatched string in tests * WIP * Lint * Add Expectation._get_examples_from_json to load from tests/test_definitions * Update checklist messages to match docs * Add 'requirements' attribute to AugmentedLibraryMetadata (already on all Expectations) * Add more helper methods to ExpectationDiagnostics and use in Expectation._get_maturity_checklist * Change temp var name when iterating over test_data_cases * Address PR feedback * Run expectation tests as a part of diagnostics Also use newer "input" and "output" over "in" and "out" in evaluate_json_test and evaluate_json_test_cfe * Remove chunks of temp commented out code and random print calls * Have build_gallery.py dump checklist output for expectations to file * Reformat with black * Flake8 (comparison to False) * isort (fix import sorting/formatting) * Only include sqlite if there are no dialects for SQL in generate_expectation_tests * Change wording of 'full suite of tests' checklist message * Add 'raise_exceptions_for_backend' kwarg to build_test_backends_list * Update build_gallery.py script to print the checklist and write to a new JSON file * Update deploy_gallery stage in pipeline to include all 3 DB service containers * Add calling build_package_gallery.py to build_gallery job * Add this branch to trigger section (temporarily) * Add scripts to configure mysql/mssql in the build_gallery job * Add some temp debugging output for CI/CD troubleshooting * Catch FileNotFoundError when os.listdir fails * Try installing GE with --editable * Lint this temp code that is going to get deleted soon anyway... * Update generate_expectation_tests to try all sql dialects if specific test_backends aren't defined * Update build_gallery job to fetch GCP creds and set env vars * Update test__get_test_results * Make sure Task reference is in format <NAME>@<MAJOR_VERSION> * Catch NotADirectoryError * Just use os.path.isdir to see if _TEST_DEFS_DIR exists * Don't try bigquery for run_diagnostics (called by build_gallery.py) * Remove the CI/CD print debugging * Use ubuntu-18.04 for deploy_gallery stage * Update ExpectationDiagnostics.to_json_dict to add 'execution_engines_list' field * Update ExpectationDiagnostics._check_linting to just say linting fails for now * Reformat with black * Update azure-pipelines.yml to not run the deploy_gallery stage on this feature branch * Update a doc link that pointed to a file that no longer exists Co-authored-by: Ken Wade <[email protected]> * Fix link causing docusaurus build to fail * commenting out statement renderers from sidebar * Delete column_map_equal_three_example.py * removing maturity as a required key in library metadata + linting * Update contrib expectations to use print_diagnostic_checklist method (#4247) * fixed some broken links Co-authored-by: Edward Su <[email protected]> Co-authored-by: Abe Gong <[email protected]> Co-authored-by: Abe <[email protected]> Co-authored-by: Ken Wade <[email protected]> Co-authored-by: Don Heppner <[email protected]> Co-authored-by: kenwade4 <[email protected]> Co-authored-by: Rachel-Reverie <[email protected]> Co-authored-by: Chetan Kini <[email protected]> Co-authored-by: Alex Sherstinsky <[email protected]>
Rachel-Reverie
added a commit
that referenced
this pull request
Feb 25, 2022
* Create how_to_enable_additional_execution_engines_for_custom_expectations.md add documentation for implementing metrics in SQLAlchemy and Spark * initial commit of custom expectations docs restructure * clean up structure and titles * put creating expectation doc scripts under test * - Reworks custom expectation doc into a more explicit guide for custom column aggregate expectations - Fixes pointers in other docs to this doc - Brings supporting example into great_expectations from superconductive/ge_tutorials * cleanup on column_aggregate docs | skeleton for column_map doc * fixed references to map_metric * moving reference file under tests, fixing references * Rename custom_column_max_example.py * Restore colors to `contributing maturity.md` (#3910) * Draft overview for custom_expectations (#3911) * Draft Overview for custom_expectations * Add creating_custom_expectations/components/prerequisites.jsx * Feature/expectation completeness checklist (#3906) * Implement the first several steps of generate_diagnostic_checklist, with tests and unit-testable supporting methods. * Refactor tests so that test_run_diagnostics uses the same test cases as test_generate_diagnostic_checklist * Move expect_column_values_to_equal_three.py to a fixtures directory * expansion of contributing_maturity with specifics for expectations * Updates and edits to custom expectations overview and contributing maturity docs * progress on adapting column agg doc to fit pattern from column map doc | edits to template and example to support | linting * rework of agg doc re: map doc | updates to supporting code | linting and typo fixes * updates to agg doc | small edits to metrics & prereqs * Feature/refactor and type expectation diagnostic report (#3929) * Initial scaffolding. One type and one test * Scaffold most of the rest of the objects we'll need, with some tests. * Add tests to unpack the behavior of @DataClass combined with SerializableDotDict * More WIP, especially tests * A spate of renaming and tightening * Add SerializableDotDict.keys * Update expectation_test_data_cases to use SerializableDictDot * Fix gtypos * Update SerializableDictDot, with tests * The meat of the refactor, mostly done with some WIP * Fixed test__count_positive_and_negative_example_cases * Decruft * Fix test_include_in_gallery_flag * Make .to_dict recursive * Linting * Remove pydantic; Remove submessages in execution_engines; test_generate_diagnostic_checklist__first_iteration now passes * Refactor _count_unexpected_cases_and_get_sub_messages into _count_unexpected_cases * Fix docstring * Clean up types and methods around LibraryMetadata * Add support for enum serialization * Implement LegacyAugmentedLibraryMetadataAdapter; Tidy up a little * Minor bugfixes. Many docstrings * Add docstrings for test_data_cases * Improve _get_metric_diagnostics_list * Clean up _get_test_results * Work on docstrings and more bug fixes. Re-up pydantic. Need to get out of the weeds soon. * Deprecate LegacyAugmentedLibraryMetadataAdapter * Tidy up rendering methods * Add RendererTestDiagnostics object * Add unit tests for _get_augmented_library_metadata and _get_examples * Unit test for _get_description_diagnostics * Implement OtherExpectationTestDiagnostics * Flow typed ExecutedExpectationTestCases through * Remove dependence on global _registered_renderers. Get another test to pass. * Clean up tests and get business logic for Has at least one positive and negative example case, and all test cases pass working * Decruft. Skip one failing test * Feature/changes from novuum jan3 review (#3941) * Add ExpectationDiagnosticMaturityMessages class and refactor around it * Add optional AugmentedLibraryMetadata.package and ExpectationDiagnosticCheckMessage.doc_url * Decruft * Fix mismatched string in tests * Delete test_expectation__get_execution_engine_dict _get_execution_engine_dict was replaced by _get_execution_engine_diagnostics * Update tests/expectations/test_expectation_diagnostics.py Co-authored-by: Don Heppner <[email protected]> * Implement ExpectationDiagnostics.to_json_dict method and use in build_gallery.py script * Update build_gallery.py script to collect expectation tracebacks and display before writing gallery json file * Allow ExpectationTestDataCases to receive 'schemas' * Add commented-out 'test_backends' attribute to ExpectationTestDataCases class * Add a mysql docker image that works with M1 Macs % docker-compose up Pulling mysql_db (mysql:8.0.20)... 8.0.20: Pulling from library/mysql ERROR: no matching manifest for linux/arm64/v8 in the manifest list entries * Ensure that the 'configuration' object passed to renderers are typed objects * Update build_gallery.py script to not die if there are expectation tracebacks * Use dot notation for args passed to ExpectationDiagnosticCheckMessage Co-authored-by: Ken Wade <[email protected]> Co-authored-by: Don Heppner <[email protected]> * [FEATURE] Add maturity_checklist to ExpectationDiagnostics (#4061) * Add maturity_checklist to ExpectationDiagnostics and use in Expectation.run_diagnostics * Remove concept_only from ExpectationDiagnosticMaturityMessages * Re-format files with black * Update the 'broken links' identified by docs_link_checker.py script * [DOCS] WIP Docs/custom expectation test cases (#3969) * updated opening paragraph * further edits * completed ex_cases doc | re-orged | edited sidebar & example file to support * fixed codeblock line refs * Update docs/guides/expectations/features_custom_expectations/how_to_add_example_cases_for_an_expectation.md Co-authored-by: Rachel-Reverie <[email protected]> * Update docs/guides/expectations/features_custom_expectations/how_to_add_example_cases_for_an_expectation.md Co-authored-by: Rachel-Reverie <[email protected]> * Update docs/guides/expectations/features_custom_expectations/how_to_add_example_cases_for_an_expectation.md Co-authored-by: Rachel-Reverie <[email protected]> * Apply suggestions from code review Co-authored-by: Rachel-Reverie <[email protected]> Co-authored-by: Rachel-Reverie <[email protected]> * removed errant print | removed maturity tag * removed errant print | removed maturity tag * fixed maturity verbiage * fixed code refs | fixed diagnostics verbiage | expanded contribution stub * added details on testing for errors * fix to diagnostics verbiage * [BUGFIX] Pin `pip --upgrade` to a specific version to bypass bug in pip's underlying algorithm (#4100) * fix: pin pip upgrade * chore: test change to trigger pipeline * fix: also pin dependency graph pip upgrades * chore: revert changes * [BUGFIX] Use `pip==20.2.4` for usage statistics stage of CI/CD (#4102) * fix: pin pip upgrade * chore: test change to trigger pipeline * fix: also pin dependency graph pip upgrades * chore: revert changes * chore: revert usage stats pip versions * [MAINTENANCE] Fix Configuration Schema and Refactor Rule-Based Profiler; Initial Implementation of Reconciliation Logic Between Configuration and Runtime Arguments (#4088) * [BUGFIX] Fix shared state issue in renderer test (#4000) * chore: add assert to test problem * test: use abspath to find JSON files * test: add UNSUPPORTED_EXPECTATIONS * chore: add comment to explain test * refactor: remove defaultdict * chore: update comment * fix: bring back defaultdict * chore: fix typo * chore: update comments * fix: use context manager pattern for cd * chore: move certain asserts out of context manager scope * refactor: use context manager cd for tests * chore: remove unnecessary imports * test: fix test identation * refactor: split up os join usage * bringing example code for agg doc under test * removing as a required key for library_metadata * replaced deprecated alias * replaced deprecated alias * [FEATURE] Add classes for Great Expectations contrib packaging JSON object (#3922) * feat: add JSON object classes * feat: add script to JSON parse * chore: update enum value per @abegong * refactor: move types into CLI tool * feat: add diagnostic parsing support to package class * feat: implement dynamic importing to retrieve Expectations * refactor: use issubclass checks * feat: misc cleanup to ensure proper serialization * fix: misc fixes to get hook working * chore: update logger calls * chore: add example Package JSON * refactor: use enum for social media types * feat: make dataclasses frozen * chore: make to_json_file func private * chore: remove unnecessary imports * refactor: rename types to package * chore: add better exception/errors * chore: misc comments * refactor: move serialization methods out of class * refactor: use Config suffix * refactor: remove Config in place of Manifest * fix: rename config to manifest * [DOCS] Adds Input Validation / Type Checking doc for custom expectations (#4092) * implement validate_configuraton method * typo fixes * typo fixes * fleshed out input_validation doc * Update docs/guides/expectations/features_custom_expectations/how_to_add_input_validation_for_an_expectation.md Co-authored-by: Rachel-Reverie <[email protected]> * edits for content and clarity * Update how_to_add_input_validation_for_an_expectation.md Co-authored-by: Rachel-Reverie <[email protected]> * WIP Docs/how to create custom column map expectations WIP (#3914) * * Rough draft text for How to create a custom Column map Expectation * Add How to create a Custom column Map Expectation to sidebar.js * Simplify the template for column map expectations. * Begin to implement generate_diagnostic_checklist, with tests. WIP * Drop extraneous print statement in self_check/util.py. * Refactor generate_diagnostic_checklist to return a string; Get tests working * Factor out _convert_checks_into_output_message and add tests * Implement logic for Check whether core logic for this Expectation exists and passes tests on at least one Execution Engine * Refactor tests so that test_run_diagnostics uses the same test cases as test_generate_diagnostic_checklist * Factor out and test _count_positive_and_negative_example_cases * Add a good docstring test * Factor out _count_unexpected_cases_and_get_sub_messages * Revert sidebars.js * Revert three files that shouldn't have been on this branch to begin with * Update how_to_create_custom_column_map_expectations.md with draft text. Update corresponding Expectation template. * black * isort * Update changelog * Improve template * Improve code style guide * Full text for the first two sections. Make step numbers sane. * Add instructions for example cases. * Drop tests/expectations/expect_column_values_to_equal_three.py * * Clean up create_custom_expectations/components/prerequisites.jsx * Still writing and writing * finish draft text, with corresponding changes to the template file * First friction log * Tidy up Other Parameters * Add integration test fixture * Fixes codeblock references in guide | Brings template & example file in line with previously in-line codeblocks | Minor formatting, linting, typo fixes * unifying completeness checklist blocks * typo fixes * fixing broken links * linting * edits to map doc & prereqs * added block addressing _validate * edits to template/example | fixes to code refs | expansions to testing & contribution | fixes to diagnostics verbiage Co-authored-by: Austin Robinson <[email protected]> Co-authored-by: Austin Ziech Robinson <[email protected]> Co-authored-by: Ken Wade <[email protected]> * bringing map example code under test * fixed code refs * fixed code refs * fixed code refs * [DOCS] Adds docs for implementing Spark & SQLAlchemy backends for Custom Expectations (#4162) * updated example scripts with spark/sqlalchemy backends * Complete SQLAlchemy Doc * minor verbiage changes * Complete Spark Doc * fixed code refs * [FEATURE] Add parsing of Expectation diagnostics to contrib packaging JSON object (#4114) * feat: add JSON object classes * feat: add script to JSON parse * chore: update enum value per @abegong * refactor: move types into CLI tool * feat: add diagnostic parsing support to package class * feat: implement dynamic importing to retrieve Expectations * refactor: use issubclass checks * feat: misc cleanup to ensure proper serialization * fix: misc fixes to get hook working * chore: update logger calls * chore: add example Package JSON * refactor: use enum for social media types * feat: make dataclasses frozen * chore: make to_json_file func private * chore: remove unnecessary imports * refactor: rename types to package * chore: add better exception/errors * chore: misc comments * refactor: move serialization methods out of class * refactor: use Config suffix * refactor: remove Config in place of Manifest * fix: rename config to manifest * feat: start parsing diagnostics * feat: parse requirements * feat: update requirements parsing * refactor: make manifest mutable * feat: continue impl * feat: continue update attr feature * test: add tests for update methods * chore: revert files back to hackathon-docs state * fix: push unlinted changes * chore: revert docs * chore: revert azure yml * chore: delete extraneous files * chore: revert additional files * chore: revert final files * test: add test for static attrs * chore: revert cli.py * feat: exit early with bad requirements path * feat: ensure enums are JSON serializable * feat: add sync cmd * chore: update comments * chore: additional cleanup * chore: misc updates * feat: add parsing support for package_info.yml * test: start writing tests for addl parsing * fix: fix filenotfound error issue * feat: write build script for CI/CD uploading to S3 * chore: misc updates * chore: misc cleanup * chore: bump black requirement * [DOCS] "How to contribute a Great Expectations package" (#3932) * docs: create how-to-guide file * docs: start guide * docs: write first version of doc * updated contributing_package & supporting files * style fixes; * docs: update doc * Update contributing_package.md Split grouped terminal commands into separate instructions Co-authored-by: Austin Robinson <[email protected]> Co-authored-by: Austin Ziech Robinson <[email protected]> Co-authored-by: Rachel-Reverie <[email protected]> * import fix * fixing broken links * fix: update import statement to ensure CI/CD script works * fix: ensure that build script picks up contrib packages * fix: ensure CI/CD script works from scripts/ and project root * [DOCS] How to add Statement Renderers to Custom Expectations (#4236) * new intro * renaming, links * initial work * implementing renderers for example scripts * statement renderer doc * typo fix * linting and code ref fixes * fixing code refs * fixing code refs * fixing code refs * Fixing code refs * fixing code refs * [DOCS] How to contribute a Custom Expectation (#4220) * reorg, fixes to pointers & sidebar * started on doc update * first pass rework * content and link fixes * added reqs * created custom expectation targeted contribution doc * linking fixes * reorg and linking fixes * Update expectations.md * commenting out unused docs from sidebar * fixes re: 02/09 rev * fixes re: 02/09 rev * fixes re: 02/09 rev * [DOCS] Updates Custom Expectations Overview (#4237) * linking and verbiage * fixes to linting doc * [DOCS] Docs/creating custom expectations/overview (#4238) * linking and verbiage * fixes to linting doc * linking * [MAINTENANCE] Refactor generate_expectation_tests and run Expectation tests as part of diagnostics (#3951) * Initial scaffolding. One type and one test * Scaffold most of the rest of the objects we'll need, with some tests. * Add tests to unpack the behavior of @DataClass combined with SerializableDotDict * More WIP, especially tests * A spate of renaming and tightening * Add SerializableDotDict.keys * Update expectation_test_data_cases to use SerializableDictDot * Fix gtypos * Update SerializableDictDot, with tests * The meat of the refactor, mostly done with some WIP * Fixed test__count_positive_and_negative_example_cases * Decruft * Fix test_include_in_gallery_flag * Make .to_dict recursive * Linting * Remove pydantic; Remove submessages in execution_engines; test_generate_diagnostic_checklist__first_iteration now passes * Refactor _count_unexpected_cases_and_get_sub_messages into _count_unexpected_cases * Fix docstring * Clean up types and methods around LibraryMetadata * Add support for enum serialization * Implement LegacyAugmentedLibraryMetadataAdapter; Tidy up a little * Small rename * Remove pydantic * Remove pydantic * Remove pydantic * Minor bugfixes. Many docstrings * Add docstrings for test_data_cases * Improve _get_metric_diagnostics_list * Clean up _get_test_results * Work on docstrings and more bug fixes. Re-up pydantic. Need to get out of the weeds soon. * Deprecate LegacyAugmentedLibraryMetadataAdapter * Tidy up rendering methods * Add RendererTestDiagnostics object * Add unit tests for _get_augmented_library_metadata * Add unit tests for _get_augmented_library_metadata and _get_examples * Unit test for _get_description_diagnostics * Implement OtherExpectationTestDiagnostics * Flow typed ExecutedExpectationTestCases through * Remove dependence on global _registered_renderers. Get another test to pass. * Get another test to pass * Fix another test * Fix another test * Clean up tests and get business logic for Has at least one positive and negative example case, and all test cases pass working * Decruft. Skip one failing test * black * isort * Comment out pydantic * isort and black * Lint manually * Lint manually * Feature/changes from novuum jan3 review (#3941) * Add ExpectationDiagnosticMaturityMessages class and refactor around it * Add optional AugmentedLibraryMetadata.package and ExpectationDiagnosticCheckMessage.doc_url * Decruft * Linting * Fix mismatched string in tests * WIP * Lint * Add Expectation._get_examples_from_json to load from tests/test_definitions * Update checklist messages to match docs * Add 'requirements' attribute to AugmentedLibraryMetadata (already on all Expectations) * Add more helper methods to ExpectationDiagnostics and use in Expectation._get_maturity_checklist * Change temp var name when iterating over test_data_cases * Address PR feedback * Run expectation tests as a part of diagnostics Also use newer "input" and "output" over "in" and "out" in evaluate_json_test and evaluate_json_test_cfe * Remove chunks of temp commented out code and random print calls * Have build_gallery.py dump checklist output for expectations to file * Reformat with black * Flake8 (comparison to False) * isort (fix import sorting/formatting) * Only include sqlite if there are no dialects for SQL in generate_expectation_tests * Change wording of 'full suite of tests' checklist message * Add 'raise_exceptions_for_backend' kwarg to build_test_backends_list * Update build_gallery.py script to print the checklist and write to a new JSON file * Update deploy_gallery stage in pipeline to include all 3 DB service containers * Add calling build_package_gallery.py to build_gallery job * Add this branch to trigger section (temporarily) * Add scripts to configure mysql/mssql in the build_gallery job * Add some temp debugging output for CI/CD troubleshooting * Catch FileNotFoundError when os.listdir fails * Try installing GE with --editable * Lint this temp code that is going to get deleted soon anyway... * Update generate_expectation_tests to try all sql dialects if specific test_backends aren't defined * Update build_gallery job to fetch GCP creds and set env vars * Update test__get_test_results * Make sure Task reference is in format <NAME>@<MAJOR_VERSION> * Catch NotADirectoryError * Just use os.path.isdir to see if _TEST_DEFS_DIR exists * Don't try bigquery for run_diagnostics (called by build_gallery.py) * Remove the CI/CD print debugging * Use ubuntu-18.04 for deploy_gallery stage * Update ExpectationDiagnostics.to_json_dict to add 'execution_engines_list' field * Update ExpectationDiagnostics._check_linting to just say linting fails for now * Reformat with black * Update azure-pipelines.yml to not run the deploy_gallery stage on this feature branch * Update a doc link that pointed to a file that no longer exists Co-authored-by: Ken Wade <[email protected]> * Fix link causing docusaurus build to fail * commenting out statement renderers from sidebar * Delete column_map_equal_three_example.py * removing maturity as a required key in library metadata + linting * Update contrib expectations to use print_diagnostic_checklist method (#4247) * fixed some broken links Co-authored-by: Edward Su <[email protected]> Co-authored-by: Abe Gong <[email protected]> Co-authored-by: Abe <[email protected]> Co-authored-by: Ken Wade <[email protected]> Co-authored-by: Don Heppner <[email protected]> Co-authored-by: kenwade4 <[email protected]> Co-authored-by: Rachel-Reverie <[email protected]> Co-authored-by: Chetan Kini <[email protected]> Co-authored-by: Alex Sherstinsky <[email protected]>
austiezr
added a commit
that referenced
this pull request
Feb 25, 2022
* release candidate for 0.14.6 (#4172) * dataprofiler expectations * submodule initial pass * clear __init__ * refactor to new contrib paradigm * add expert name * function type hinting for mypy * dataprofiler expectations * submodule initial pass * [FEATURE] Allow Data Docs to be rendered in night mode (#4130) * add dark mode themeing to data docs pages * darkmode logo * remove extra css file and move logo to CDN Co-authored-by: talagluck <[email protected]> * clear __init__ * [FEATURE] SimpleDateFormatParameterBuilder (#4156) * Implement Class SimpleDateFormatStringParameterBuilder * Update changelog.rst (#4163) Modify the 0.10.8 entry to reference the proper environment variable for custom jupyter command (GE_JUPYTER_CMD vs GE_JUPYTER_COMMAND) * [FEATURE] Introduce Top-Level Abstract "ConfigPeer" class for handling configuration output for Marshmallo Schema validated Config objects (#4183) * [MAINTENANCE] Make `cli_integration` stage in primary `great_expectations` pipeline run in parallel (#4187) * feat: add cron schedule for main pipeline * feat: write update YAML for pipelines * chore: test change * fix: add postgres to new stage * chore: remove new stage * chore: add comments to YAMLs * chore: revert test change * chore: test change * chore: update pip * chore: revert test changes * chore: make always true * refactor: use Azure checks instead of scope_check stage * feat: remove test_cli prereqs * chore: revert test changes * chore: test change * chore: try making usage_stats last * fix: remove cycle from dependsOn * refactor: move usage_stats to end of YAML * fix: update cli stage ref * feat: parallelize cli_integration * [FEATURE] Update `suite new --profile` to work with Rule Based Profiler (#4171) * chore: type hints * chore: add notes * refactor: misc cleanup * chore: revert CLI signature * refactor: more cleanup * chore: revert change to test util * feat: add profiler_name as valid arg to --profile * chore: bump requirements.txt * feat: configure optional option arg * feat: start modifying notebooks * test: fix broken tests * chore: revert requirements * chore: type hint and add comment * feat: update renderer logic * test: add CLI tests * test: write tests for renderer * feat: add CLI messaging about profiler type * [MAINTENANCE] Refactor RuleBasedProfiler toolkit pattern (#4191) * docs passing sqlalchemy.create_engine kwargs through connection_string Added to docs in docs/guides/connecting_to_your_data/database/*.md; explains how to pass custom params via use of `connection_string` passing kwargs. * refactor: remove toolkit in place of static methods * chore: make context helper method private * chore: revert docs * fixes two references to the Getting Started tutorial (#4189) * [MAINTENANCE] Revert changes to `dependency_graph` pipeline * [MAINTENANCE] Refactor relative imports (#4195) * refactor: start cleanup * chore: revert to dev * refactor: fix all relative imports * [MAINTENANCE]: Remove temp file accidentally committed (#4201) * refactor to new contrib paradigm * [FEATURE] Support Multi-Dimensional Metric Computations Generically for Multi-Batch Parameter Builders (#4206) * [MAINTENANCE] Update default candidate strings SimpleDateFormatString parameter builder (#4193) * [MAINTENANCE] minor type hints clean up (#4214) * [MAINTENANCE] RBP testing framework changes (#4184) * push of proposed refactor * Merge branch 'develop' into feature/GREAT-494/refactor-rbp-tests * develop: [FEATURE] SimpleDateFormatParameterBuilder (#4156) * tests pass * Update test_user_workflow_fixtures.py * Update conftest.py * bugfix * pushing refactor after review * final update * pushing recent changes * [BUGFIX] Update validate_configuration for core Expectations that don't return… (#4216) * Update validate_configuration for core Expectations that don't return a value * Annotate return type of validate_configuration method on Expectations that have it * add conditional check for 'expect_column_values_to_be_in_type_list' (#4200) * add conditional check for column * fix parallel bug * lint * [BUGFIX] Datadocs filter on date is broken (#4217) * update date filter function * update lib version and fix formatter * update linting and fix pull request template to use new docs * update the changelog * Bugfix/devrel 189/datadocs filter on date is broken (#4218) * update date filter function * update lib version and fix formatter * update linting and fix pull request template to use new docs * update the changelog * update changelog in the right place * [ENHANCEMENT] Allow users to pass in any set of polygon points in expectation for point to be within region (#2520) * Create expectation file expect record point to be within geo region * Initial expectation setup * Implement pandas expectation * Add in negative test and read in data as class variable, so doesn't load on every expectation invocation * Rename package to be consistent with Expectation class * black formatter * isort * Add parameter polygon_points in _pandas function * Update input variable definitions to expect polygon_points, and update tests for passing in input latitude-longitude points to expectation * Update documentation given addition of support for passing in polygon points * Add new test * New polygon points for the USA test * linting Co-authored-by: DXcarlos <[email protected]> Co-authored-by: anthonyburdi <[email protected]> Co-authored-by: Austin Robinson <[email protected]> * [ENHANCEMENT] Better support Hive, better support BigQuery. (#2624) * Add hive to the dataset model * Add support for project id in bigquery datasources If the credentials do not have the same project as the table to be queried, the project id needs to be specified. * linting Co-authored-by: Austin Robinson <[email protected]> * [DOCS] Deepnote Deployment Pattern Guide (#4169) * first half of deepnote deployment guide * first half of deployment guide * first pass at DN deployment pattern * update sidebar.js * fixed img refs * typo fixes & codeblock linting * codeblock fix * replaced images * move process_evaluation_parameters into conditional (#4109) * [MAINTENANCE] Type hint usage statistics (#4226) * [FEATURE] Add support for sqlalchemy-bigquery while falling back on pybigquery (#4182) * Add support for sqlalchemy-bigquery while falling back on pybigquery * Black reformatting * Update package to pip install in docs * Update doc url in credentials_snippet * Feature/add support for sqlalchemy bigquery + geo columns (#4222) * updated bigquery import path * add warning to expectation if geometry package isnt installed * updated dev reqs * lint * update reqs-dev * update bigquery import path * updated import paths * lint * Use warnings.warn for DeprecationWarning about using pybigquery Co-authored-by: Ken Wade <[email protected]> * Add comment in requirements-dev-sqlalchemy.txt for sqlalchemy-bigquery Co-authored-by: Joshua Stauffer <[email protected]> * [DOCS] Deepnote Deployment Pattern Image Fixes (#4229) * first half of deepnote deployment guide * first half of deployment guide * first pass at DN deployment pattern * update sidebar.js * fixed img refs * typo fixes & codeblock linting * codeblock fix * replaced images * image linking * linking fix * release candidate for 0.14.7 (#4230) * [MAINTENANCE] Rule-Based Profiler: Fix Circular Imports; Configuration Schema Fixes; Enhanced Unit Tests; Pre-Requisites/Refactoring for Self-Estimating Expectations (#4234) * [FEATURE] Add `run_profiler_on_data` method to DataContext (#4190) * feat: add method stub * feat: start impl * chore: write docstr * test: write test stubs * feat: copy batch requests each time * refactor: make reconciliation method private * feat: finish initial version * refactor: convert batch request to dict to ensure proper serializability * feat: lay out plan after talking with Alex * feat: convert batch request to dict * feat: finish impl * feat: leverage enum check instead of isinstance * feat: add docstring and use safe_deep_copy * chore: use deepcopy instead of safe_deecopy * feat: remove copy * [FEATURE] `RegexPatternStringParameterBuilder` for `RuleBasedProfiler` (#4167) * [FEATURE] `RegexPatternStringParameterBuilder` for `RuleBasedProfiler` (#4167) Co-authored-by: Chetan Kini <[email protected]> * fix: fix issue (#4240) * remove duplicate test (#4241) * experimental column map expectation checking for vectors (#3102) * Reformat contrib expectation with black (#4244) * Improve pandas version checking in test_expectations[_cfe].py files (#4248) * [MAINTENANCE] Resolve cyclic import issue with usage stats (#4251) * chore: move imports to be in-line * chore: add comment * fix: fix cfg setting * [FEATURE] Pre-requisites in Rule-Based Profiler component relevant for Self-Estimating Expectations: customizable/flexible configuration reconciliation at Variables, DomainBuilder, ParameterBuilder, and ExpectationConfigurationBuilder levels; additional method accessors. (#4242) * [BUGFIX] Ensure `test_script_runner.py` actually raises AssertionErrors correctly (#4239) * Update test_script_runner.py * Update gcp_deployment_patterns_file_gcs_yaml_configs.py * bugfix * update this with actual values * Update multi_batch_rule_based_profiler_example.py * Update multi_batch_rule_based_profiler_example.py * clean up * [MAINTENANCE] Additional refactor to clean up cyclic imports in usage stats (#4256) * chore: move imports to be in-line * chore: add comment * fix: fix cfg setting * refactor: move away from init and use full paths * refactor: remove imports in init * [MAINTENANCE] Rule-Based Profiler prerequisite: fix quantiles profiler configuration and add comments (#4255) * [FEATURE] Add optional parameter `condition` to DefaultExpectationConfigurationBuilder (#4246) - Conditional logic parser using pyparsing added to DefaultExpectationConfigurationBuilder - Testing for DefaultExpectationConfigurationBuilder * [MAINTENANCE] Introspect Batch Request Dictionary for its kind and instantiate accordingly (#4259) * [MAINTENANCE] Minor clean up in style of an RBP test fixture; making variables access more robust (#4261) * add expert name * Check for pandas>=024 not pandas>=24 (#4263) * [BUGFIX] Add support for SqlAlchemyQueryStore connection_string credentials (#4224) * Add support for SqlAlchemyQueryStore connection_string credentials * Add test for SqlAlchemyQueryStore connection_string credentials * Fix formatting * Fix formatting * [DOCS/FEATURE] Hackathon Contribution Docs (#3897) * Create how_to_enable_additional_execution_engines_for_custom_expectations.md add documentation for implementing metrics in SQLAlchemy and Spark * initial commit of custom expectations docs restructure * clean up structure and titles * put creating expectation doc scripts under test * - Reworks custom expectation doc into a more explicit guide for custom column aggregate expectations - Fixes pointers in other docs to this doc - Brings supporting example into great_expectations from superconductive/ge_tutorials * cleanup on column_aggregate docs | skeleton for column_map doc * fixed references to map_metric * moving reference file under tests, fixing references * Rename custom_column_max_example.py * Restore colors to `contributing maturity.md` (#3910) * Draft overview for custom_expectations (#3911) * Draft Overview for custom_expectations * Add creating_custom_expectations/components/prerequisites.jsx * Feature/expectation completeness checklist (#3906) * Implement the first several steps of generate_diagnostic_checklist, with tests and unit-testable supporting methods. * Refactor tests so that test_run_diagnostics uses the same test cases as test_generate_diagnostic_checklist * Move expect_column_values_to_equal_three.py to a fixtures directory * expansion of contributing_maturity with specifics for expectations * Updates and edits to custom expectations overview and contributing maturity docs * progress on adapting column agg doc to fit pattern from column map doc | edits to template and example to support | linting * rework of agg doc re: map doc | updates to supporting code | linting and typo fixes * updates to agg doc | small edits to metrics & prereqs * Feature/refactor and type expectation diagnostic report (#3929) * Initial scaffolding. One type and one test * Scaffold most of the rest of the objects we'll need, with some tests. * Add tests to unpack the behavior of @DataClass combined with SerializableDotDict * More WIP, especially tests * A spate of renaming and tightening * Add SerializableDotDict.keys * Update expectation_test_data_cases to use SerializableDictDot * Fix gtypos * Update SerializableDictDot, with tests * The meat of the refactor, mostly done with some WIP * Fixed test__count_positive_and_negative_example_cases * Decruft * Fix test_include_in_gallery_flag * Make .to_dict recursive * Linting * Remove pydantic; Remove submessages in execution_engines; test_generate_diagnostic_checklist__first_iteration now passes * Refactor _count_unexpected_cases_and_get_sub_messages into _count_unexpected_cases * Fix docstring * Clean up types and methods around LibraryMetadata * Add support for enum serialization * Implement LegacyAugmentedLibraryMetadataAdapter; Tidy up a little * Minor bugfixes. Many docstrings * Add docstrings for test_data_cases * Improve _get_metric_diagnostics_list * Clean up _get_test_results * Work on docstrings and more bug fixes. Re-up pydantic. Need to get out of the weeds soon. * Deprecate LegacyAugmentedLibraryMetadataAdapter * Tidy up rendering methods * Add RendererTestDiagnostics object * Add unit tests for _get_augmented_library_metadata and _get_examples * Unit test for _get_description_diagnostics * Implement OtherExpectationTestDiagnostics * Flow typed ExecutedExpectationTestCases through * Remove dependence on global _registered_renderers. Get another test to pass. * Clean up tests and get business logic for Has at least one positive and negative example case, and all test cases pass working * Decruft. Skip one failing test * Feature/changes from novuum jan3 review (#3941) * Add ExpectationDiagnosticMaturityMessages class and refactor around it * Add optional AugmentedLibraryMetadata.package and ExpectationDiagnosticCheckMessage.doc_url * Decruft * Fix mismatched string in tests * Delete test_expectation__get_execution_engine_dict _get_execution_engine_dict was replaced by _get_execution_engine_diagnostics * Update tests/expectations/test_expectation_diagnostics.py Co-authored-by: Don Heppner <[email protected]> * Implement ExpectationDiagnostics.to_json_dict method and use in build_gallery.py script * Update build_gallery.py script to collect expectation tracebacks and display before writing gallery json file * Allow ExpectationTestDataCases to receive 'schemas' * Add commented-out 'test_backends' attribute to ExpectationTestDataCases class * Add a mysql docker image that works with M1 Macs % docker-compose up Pulling mysql_db (mysql:8.0.20)... 8.0.20: Pulling from library/mysql ERROR: no matching manifest for linux/arm64/v8 in the manifest list entries * Ensure that the 'configuration' object passed to renderers are typed objects * Update build_gallery.py script to not die if there are expectation tracebacks * Use dot notation for args passed to ExpectationDiagnosticCheckMessage Co-authored-by: Ken Wade <[email protected]> Co-authored-by: Don Heppner <[email protected]> * [FEATURE] Add maturity_checklist to ExpectationDiagnostics (#4061) * Add maturity_checklist to ExpectationDiagnostics and use in Expectation.run_diagnostics * Remove concept_only from ExpectationDiagnosticMaturityMessages * Re-format files with black * Update the 'broken links' identified by docs_link_checker.py script * [DOCS] WIP Docs/custom expectation test cases (#3969) * updated opening paragraph * further edits * completed ex_cases doc | re-orged | edited sidebar & example file to support * fixed codeblock line refs * Update docs/guides/expectations/features_custom_expectations/how_to_add_example_cases_for_an_expectation.md Co-authored-by: Rachel-Reverie <[email protected]> * Update docs/guides/expectations/features_custom_expectations/how_to_add_example_cases_for_an_expectation.md Co-authored-by: Rachel-Reverie <[email protected]> * Update docs/guides/expectations/features_custom_expectations/how_to_add_example_cases_for_an_expectation.md Co-authored-by: Rachel-Reverie <[email protected]> * Apply suggestions from code review Co-authored-by: Rachel-Reverie <[email protected]> Co-authored-by: Rachel-Reverie <[email protected]> * removed errant print | removed maturity tag * removed errant print | removed maturity tag * fixed maturity verbiage * fixed code refs | fixed diagnostics verbiage | expanded contribution stub * added details on testing for errors * fix to diagnostics verbiage * [BUGFIX] Pin `pip --upgrade` to a specific version to bypass bug in pip's underlying algorithm (#4100) * fix: pin pip upgrade * chore: test change to trigger pipeline * fix: also pin dependency graph pip upgrades * chore: revert changes * [BUGFIX] Use `pip==20.2.4` for usage statistics stage of CI/CD (#4102) * fix: pin pip upgrade * chore: test change to trigger pipeline * fix: also pin dependency graph pip upgrades * chore: revert changes * chore: revert usage stats pip versions * [MAINTENANCE] Fix Configuration Schema and Refactor Rule-Based Profiler; Initial Implementation of Reconciliation Logic Between Configuration and Runtime Arguments (#4088) * [BUGFIX] Fix shared state issue in renderer test (#4000) * chore: add assert to test problem * test: use abspath to find JSON files * test: add UNSUPPORTED_EXPECTATIONS * chore: add comment to explain test * refactor: remove defaultdict * chore: update comment * fix: bring back defaultdict * chore: fix typo * chore: update comments * fix: use context manager pattern for cd * chore: move certain asserts out of context manager scope * refactor: use context manager cd for tests * chore: remove unnecessary imports * test: fix test identation * refactor: split up os join usage * bringing example code for agg doc under test * removing as a required key for library_metadata * replaced deprecated alias * replaced deprecated alias * [FEATURE] Add classes for Great Expectations contrib packaging JSON object (#3922) * feat: add JSON object classes * feat: add script to JSON parse * chore: update enum value per @abegong * refactor: move types into CLI tool * feat: add diagnostic parsing support to package class * feat: implement dynamic importing to retrieve Expectations * refactor: use issubclass checks * feat: misc cleanup to ensure proper serialization * fix: misc fixes to get hook working * chore: update logger calls * chore: add example Package JSON * refactor: use enum for social media types * feat: make dataclasses frozen * chore: make to_json_file func private * chore: remove unnecessary imports * refactor: rename types to package * chore: add better exception/errors * chore: misc comments * refactor: move serialization methods out of class * refactor: use Config suffix * refactor: remove Config in place of Manifest * fix: rename config to manifest * [DOCS] Adds Input Validation / Type Checking doc for custom expectations (#4092) * implement validate_configuraton method * typo fixes * typo fixes * fleshed out input_validation doc * Update docs/guides/expectations/features_custom_expectations/how_to_add_input_validation_for_an_expectation.md Co-authored-by: Rachel-Reverie <[email protected]> * edits for content and clarity * Update how_to_add_input_validation_for_an_expectation.md Co-authored-by: Rachel-Reverie <[email protected]> * WIP Docs/how to create custom column map expectations WIP (#3914) * * Rough draft text for How to create a custom Column map Expectation * Add How to create a Custom column Map Expectation to sidebar.js * Simplify the template for column map expectations. * Begin to implement generate_diagnostic_checklist, with tests. WIP * Drop extraneous print statement in self_check/util.py. * Refactor generate_diagnostic_checklist to return a string; Get tests working * Factor out _convert_checks_into_output_message and add tests * Implement logic for Check whether core logic for this Expectation exists and passes tests on at least one Execution Engine * Refactor tests so that test_run_diagnostics uses the same test cases as test_generate_diagnostic_checklist * Factor out and test _count_positive_and_negative_example_cases * Add a good docstring test * Factor out _count_unexpected_cases_and_get_sub_messages * Revert sidebars.js * Revert three files that shouldn't have been on this branch to begin with * Update how_to_create_custom_column_map_expectations.md with draft text. Update corresponding Expectation template. * black * isort * Update changelog * Improve template * Improve code style guide * Full text for the first two sections. Make step numbers sane. * Add instructions for example cases. * Drop tests/expectations/expect_column_values_to_equal_three.py * * Clean up create_custom_expectations/components/prerequisites.jsx * Still writing and writing * finish draft text, with corresponding changes to the template file * First friction log * Tidy up Other Parameters * Add integration test fixture * Fixes codeblock references in guide | Brings template & example file in line with previously in-line codeblocks | Minor formatting, linting, typo fixes * unifying completeness checklist blocks * typo fixes * fixing broken links * linting * edits to map doc & prereqs * added block addressing _validate * edits to template/example | fixes to code refs | expansions to testing & contribution | fixes to diagnostics verbiage Co-authored-by: Austin Robinson <[email protected]> Co-authored-by: Austin Ziech Robinson <[email protected]> Co-authored-by: Ken Wade <[email protected]> * bringing map example code under test * fixed code refs * fixed code refs * fixed code refs * [DOCS] Adds docs for implementing Spark & SQLAlchemy backends for Custom Expectations (#4162) * updated example scripts with spark/sqlalchemy backends * Complete SQLAlchemy Doc * minor verbiage changes * Complete Spark Doc * fixed code refs * [FEATURE] Add parsing of Expectation diagnostics to contrib packaging JSON object (#4114) * feat: add JSON object classes * feat: add script to JSON parse * chore: update enum value per @abegong * refactor: move types into CLI tool * feat: add diagnostic parsing support to package class * feat: implement dynamic importing to retrieve Expectations * refactor: use issubclass checks * feat: misc cleanup to ensure proper serialization * fix: misc fixes to get hook working * chore: update logger calls * chore: add example Package JSON * refactor: use enum for social media types * feat: make dataclasses frozen * chore: make to_json_file func private * chore: remove unnecessary imports * refactor: rename types to package * chore: add better exception/errors * chore: misc comments * refactor: move serialization methods out of class * refactor: use Config suffix * refactor: remove Config in place of Manifest * fix: rename config to manifest * feat: start parsing diagnostics * feat: parse requirements * feat: update requirements parsing * refactor: make manifest mutable * feat: continue impl * feat: continue update attr feature * test: add tests for update methods * chore: revert files back to hackathon-docs state * fix: push unlinted changes * chore: revert docs * chore: revert azure yml * chore: delete extraneous files * chore: revert additional files * chore: revert final files * test: add test for static attrs * chore: revert cli.py * feat: exit early with bad requirements path * feat: ensure enums are JSON serializable * feat: add sync cmd * chore: update comments * chore: additional cleanup * chore: misc updates * feat: add parsing support for package_info.yml * test: start writing tests for addl parsing * fix: fix filenotfound error issue * feat: write build script for CI/CD uploading to S3 * chore: misc updates * chore: misc cleanup * chore: bump black requirement * [DOCS] "How to contribute a Great Expectations package" (#3932) * docs: create how-to-guide file * docs: start guide * docs: write first version of doc * updated contributing_package & supporting files * style fixes; * docs: update doc * Update contributing_package.md Split grouped terminal commands into separate instructions Co-authored-by: Austin Robinson <[email protected]> Co-authored-by: Austin Ziech Robinson <[email protected]> Co-authored-by: Rachel-Reverie <[email protected]> * import fix * fixing broken links * fix: update import statement to ensure CI/CD script works * fix: ensure that build script picks up contrib packages * fix: ensure CI/CD script works from scripts/ and project root * [DOCS] How to add Statement Renderers to Custom Expectations (#4236) * new intro * renaming, links * initial work * implementing renderers for example scripts * statement renderer doc * typo fix * linting and code ref fixes * fixing code refs * fixing code refs * fixing code refs * Fixing code refs * fixing code refs * [DOCS] How to contribute a Custom Expectation (#4220) * reorg, fixes to pointers & sidebar * started on doc update * first pass rework * content and link fixes * added reqs * created custom expectation targeted contribution doc * linking fixes * reorg and linking fixes * Update expectations.md * commenting out unused docs from sidebar * fixes re: 02/09 rev * fixes re: 02/09 rev * fixes re: 02/09 rev * [DOCS] Updates Custom Expectations Overview (#4237) * linking and verbiage * fixes to linting doc * [DOCS] Docs/creating custom expectations/overview (#4238) * linking and verbiage * fixes to linting doc * linking * [MAINTENANCE] Refactor generate_expectation_tests and run Expectation tests as part of diagnostics (#3951) * Initial scaffolding. One type and one test * Scaffold most of the rest of the objects we'll need, with some tests. * Add tests to unpack the behavior of @DataClass combined with SerializableDotDict * More WIP, especially tests * A spate of renaming and tightening * Add SerializableDotDict.keys * Update expectation_test_data_cases to use SerializableDictDot * Fix gtypos * Update SerializableDictDot, with tests * The meat of the refactor, mostly done with some WIP * Fixed test__count_positive_and_negative_example_cases * Decruft * Fix test_include_in_gallery_flag * Make .to_dict recursive * Linting * Remove pydantic; Remove submessages in execution_engines; test_generate_diagnostic_checklist__first_iteration now passes * Refactor _count_unexpected_cases_and_get_sub_messages into _count_unexpected_cases * Fix docstring * Clean up types and methods around LibraryMetadata * Add support for enum serialization * Implement LegacyAugmentedLibraryMetadataAdapter; Tidy up a little * Small rename * Remove pydantic * Remove pydantic * Remove pydantic * Minor bugfixes. Many docstrings * Add docstrings for test_data_cases * Improve _get_metric_diagnostics_list * Clean up _get_test_results * Work on docstrings and more bug fixes. Re-up pydantic. Need to get out of the weeds soon. * Deprecate LegacyAugmentedLibraryMetadataAdapter * Tidy up rendering methods * Add RendererTestDiagnostics object * Add unit tests for _get_augmented_library_metadata * Add unit tests for _get_augmented_library_metadata and _get_examples * Unit test for _get_description_diagnostics * Implement OtherExpectationTestDiagnostics * Flow typed ExecutedExpectationTestCases through * Remove dependence on global _registered_renderers. Get another test to pass. * Get another test to pass * Fix another test * Fix another test * Clean up tests and get business logic for Has at least one positive and negative example case, and all test cases pass working * Decruft. Skip one failing test * black * isort * Comment out pydantic * isort and black * Lint manually * Lint manually * Feature/changes from novuum jan3 review (#3941) * Add ExpectationDiagnosticMaturityMessages class and refactor around it * Add optional AugmentedLibraryMetadata.package and ExpectationDiagnosticCheckMessage.doc_url * Decruft * Linting * Fix mismatched string in tests * WIP * Lint * Add Expectation._get_examples_from_json to load from tests/test_definitions * Update checklist messages to match docs * Add 'requirements' attribute to AugmentedLibraryMetadata (already on all Expectations) * Add more helper methods to ExpectationDiagnostics and use in Expectation._get_maturity_checklist * Change temp var name when iterating over test_data_cases * Address PR feedback * Run expectation tests as a part of diagnostics Also use newer "input" and "output" over "in" and "out" in evaluate_json_test and evaluate_json_test_cfe * Remove chunks of temp commented out code and random print calls * Have build_gallery.py dump checklist output for expectations to file * Reformat with black * Flake8 (comparison to False) * isort (fix import sorting/formatting) * Only include sqlite if there are no dialects for SQL in generate_expectation_tests * Change wording of 'full suite of tests' checklist message * Add 'raise_exceptions_for_backend' kwarg to build_test_backends_list * Update build_gallery.py script to print the checklist and write to a new JSON file * Update deploy_gallery stage in pipeline to include all 3 DB service containers * Add calling build_package_gallery.py to build_gallery job * Add this branch to trigger section (temporarily) * Add scripts to configure mysql/mssql in the build_gallery job * Add some temp debugging output for CI/CD troubleshooting * Catch FileNotFoundError when os.listdir fails * Try installing GE with --editable * Lint this temp code that is going to get deleted soon anyway... * Update generate_expectation_tests to try all sql dialects if specific test_backends aren't defined * Update build_gallery job to fetch GCP creds and set env vars * Update test__get_test_results * Make sure Task reference is in format <NAME>@<MAJOR_VERSION> * Catch NotADirectoryError * Just use os.path.isdir to see if _TEST_DEFS_DIR exists * Don't try bigquery for run_diagnostics (called by build_gallery.py) * Remove the CI/CD print debugging * Use ubuntu-18.04 for deploy_gallery stage * Update ExpectationDiagnostics.to_json_dict to add 'execution_engines_list' field * Update ExpectationDiagnostics._check_linting to just say linting fails for now * Reformat with black * Update azure-pipelines.yml to not run the deploy_gallery stage on this feature branch * Update a doc link that pointed to a file that no longer exists Co-authored-by: Ken Wade <[email protected]> * Fix link causing docusaurus build to fail * commenting out statement renderers from sidebar * Delete column_map_equal_three_example.py * removing maturity as a required key in library metadata + linting * Update contrib expectations to use print_diagnostic_checklist method (#4247) * fixed some broken links Co-authored-by: Edward Su <[email protected]> Co-authored-by: Abe Gong <[email protected]> Co-authored-by: Abe <[email protected]> Co-authored-by: Ken Wade <[email protected]> Co-authored-by: Don Heppner <[email protected]> Co-authored-by: kenwade4 <[email protected]> Co-authored-by: Rachel-Reverie <[email protected]> Co-authored-by: Chetan Kini <[email protected]> Co-authored-by: Alex Sherstinsky <[email protected]> * define empty sqla_bigquery object (#4249) * define empty sqla_bigquery object * update docs * revert unnecessary exception handling * Replace pybigquery with sqlalchemy-bigquery in setup.py (great_expectations[gcp]) Co-authored-by: Ken Wade <[email protected]> Co-authored-by: Don Heppner <[email protected]> Co-authored-by: kenwade4 <[email protected]> * Remove assertion (#4271) * function type hinting for mypy * update diagnostic to 0.14.8 * add dataprofiler README * fix changelog * update READM: remove pypi link * resolving issue from mostly value Co-authored-by: Ortal <[email protected]> Co-authored-by: talagluck <[email protected]> Co-authored-by: Nathan Farmer <[email protected]> Co-authored-by: Mike Metzger <[email protected]> Co-authored-by: Alex Sherstinsky <[email protected]> Co-authored-by: Chetan Kini <[email protected]> Co-authored-by: Austin Ziech Robinson <[email protected]> Co-authored-by: William Shin <[email protected]> Co-authored-by: kenwade4 <[email protected]> Co-authored-by: Joshua Stauffer <[email protected]> Co-authored-by: Ryan Lindeborg <[email protected]> Co-authored-by: DXcarlos <[email protected]> Co-authored-by: anthonyburdi <[email protected]> Co-authored-by: Austin Robinson <[email protected]> Co-authored-by: Jacob Gallagher <[email protected]> Co-authored-by: Ben Horkley <[email protected]> Co-authored-by: Lauren F <[email protected]> Co-authored-by: David van Rooij <[email protected]> Co-authored-by: Edward Su <[email protected]> Co-authored-by: Abe Gong <[email protected]> Co-authored-by: Abe <[email protected]> Co-authored-by: Ken Wade <[email protected]> Co-authored-by: Don Heppner <[email protected]> Co-authored-by: Rachel-Reverie <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Please annotate your PR title to describe what the PR does, then give a brief bulleted description of your PR below. PR titles should begin with [BUGFIX], [FEATURE], [DOCS], or [MAINTENANCE]. If a new feature introduces breaking changes for the Great Expectations API or configuration files, please also add [BREAKING]. You can read about the tags in our contributor checklist.
Changes proposed in this pull request:
After submitting your PR, CI checks will run and @ge-cla-bot will check for your CLA signature.
For a PR with nontrivial changes, we review with both design-centric and code-centric lenses.
In a design review, we aim to ensure that the PR is consistent with our relationship to the open source community, with our software architecture and abstractions, and with our users' needs and expectations. That review often starts well before a PR, for example in github issues or slack, so please link to relevant conversations in notes below to help reviewers understand and approve your PR more quickly (e.g.
closes #123).Definition of Done
Please delete options that are not relevant.
Thank you for submitting!