Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DOCS] Universal map TOC update #4292

Merged
merged 179 commits into from
Mar 7, 2022

Conversation

Rachel-Reverie
Copy link
Contributor

Please annotate your PR title to describe what the PR does, then give a brief bulleted description of your PR below. PR titles should begin with [BUGFIX], [FEATURE], [DOCS], or [MAINTENANCE]. If a new feature introduces breaking changes for the Great Expectations API or configuration files, please also add [BREAKING]. You can read about the tags in our contributor checklist.

Changes proposed in this pull request:

  • Reorganizes the ToC according to the "Setup, Connect to data, Create Expectations, Validate data" workflow.
  • Adds technical term pages for all items in the glossary
  • Updates glossary to link to technical term pages

After submitting your PR, CI checks will run and @ge-cla-bot will check for your CLA signature.

For a PR with nontrivial changes, we review with both design-centric and code-centric lenses.

In a design review, we aim to ensure that the PR is consistent with our relationship to the open source community, with our software architecture and abstractions, and with our users' needs and expectations. That review often starts well before a PR, for example in github issues or slack, so please link to relevant conversations in notes below to help reviewers understand and approve your PR more quickly (e.g. closes #123).

Definition of Done

Please delete options that are not relevant.

  • I have made corresponding changes to the documentation

Thank you for submitting!

Rachel-Reverie and others added 30 commits December 30, 2021 10:06
… that imagery, and css for creating borderless and/or centered tables.
* Added universal map resources: imagery, mdx files for quickly loading that imagery, and css for creating borderless and/or centered tables.

* Added setup_overview.md

* Added the "Step 1: Setup" section to the ToC, along with related How To guides and the overview page for setup.

* Added a space insisde of a comment so we could have a commit to trigger a netlify build.

* Moved setup_overview.md into `guides/setup`
Updated sidebars.js
Removed broken links from setup_overview.md

* Wording change in setup_overview.md intro and wrapping up paragraphs.
Added index pages for Setup how to guides.
Linked to index pages for Setup how to guides in setup_overview.md and sidebars.js

* Update docs/guides/setup/setup_overview.md

Co-authored-by: Abe Gong <[email protected]>

* Update docs/guides/setup/setup_overview.md

Co-authored-by: Abe Gong <[email protected]>

* Update docs/guides/setup/setup_overview.md

Co-authored-by: Abe Gong <[email protected]>

* Updated wording in the prerequisites section

* Changed references to "Metadata Stores" in the Setup documentation to "Stores"

* corrected typo, better phrasing for last line of Wrapping up

* Minor change in wording in the wrapping up section.

* Update setup_overview.md

Corrected minimum version of Python required.

* Moved how to guides' category indexes for Step 1: Setup to the end of each subsection in the ToC.

* Moved how to guides' category indexes for Step 1: Setup (Data Docs and Stores sections) to the end of each subsection in the ToC.

* Correction to header levels for sub-headings and sub-sub-headings of The Setup Process.

Co-authored-by: Abe Gong <[email protected]>
- Added index page for "Step 3: Create Expectations" how to guides.
- Added placeholder page for "Step 3: Overview" so that corresponding links could be added. (Guidance on content is pending)
* Partial rough draft of overview.md for the Connect to Data step.

* More fleshed out rough draft of overview.md

* complete rough draft for overview.md

* Minor edits to overview.md
Added overview.md and corresponding how to documents into the ToC

* Partial rough draft of overview.md for the Connect to Data step.

* More fleshed out rough draft of overview.md

* complete rough draft for overview.md

* rebased on parent branch

* Nested Connect to Data guides under How to guides in the ToC

* Starting updates on index pages for the Connect to Data how to documents.

* Update docs/guides/connecting_to_your_data/overview.md

Co-authored-by: talagluck <[email protected]>

* Update docs/guides/connecting_to_your_data/overview.md

Co-authored-by: talagluck <[email protected]>

* Extended section on `test_yaml_config()`

* Removed sections of the ToC that were moved into "1. Setup" and "2. Connecting to Data".

* Added `batch_identifiers` as a potential key to replace <additional_keys_based_on_source_data_systems> in the examples when the source data system is a filesystem.

* Updated `InferredAssetDataConnector` section to include that `InferredAssetSqlDataConnector` provides similar functionality for SQL based source data systems.

* Added index for Step 2: Connect to Data to sidebar.js; updated contents of index to correspond to folder structure in sidebar.js

* Updated relative links for index.md

* Update docs/guides/connecting_to_your_data/overview.md

Co-authored-by: Abe Gong <[email protected]>

* Update docs/guides/connecting_to_your_data/overview.md

Co-authored-by: Abe Gong <[email protected]>

* - Changed heading of section 1. Boilerplate to 1. Prepare scaffolding
- Refactored name of overview file to match naming convention of other step's overview files.
- Updated internal links and references to the refactored overview file's name.

* - Corrected spelling of `test_yaml_config()`

Co-authored-by: talagluck <[email protected]>
Co-authored-by: Abe Gong <[email protected]>
* - Reorganized ToC for "Step 4: Validate Data" contents.
- Added index page for "Step 4L Validate Data" how to guides.
- Added placeholder page for "Step 4: Overview" so that corresponding links could be added. (Guidance on content is pending)

* Added universal map graphical outline to Validate Data: Overview
…tep 2: Connect to Data how to guides.

- Standardized Index headers
* First draft content for the Create Expectations: Overview page.

* Update docs/guides/expectations/create_expectations_overview.md

Co-authored-by: William Shin <[email protected]>

* Update docs/guides/expectations/create_expectations_overview.md

Co-authored-by: talagluck <[email protected]>

* Update docs/guides/expectations/create_expectations_overview.md

Co-authored-by: talagluck <[email protected]>

* Update docs/guides/expectations/create_expectations_overview.md

Co-authored-by: talagluck <[email protected]>

* Update docs/guides/expectations/create_expectations_overview.md

Co-authored-by: talagluck <[email protected]>

* Update docs/guides/expectations/create_expectations_overview.md

I'm fine with calling it the profiler workflow.  I'll make that change as well.

Co-authored-by: talagluck <[email protected]>

* Update docs/guides/expectations/create_expectations_overview.md

Co-authored-by: talagluck <[email protected]>

* Update docs/guides/expectations/create_expectations_overview.md

I think this needs to be "When following one of" but I'll make that edit after committing these changes.

Co-authored-by: talagluck <[email protected]>

* Update docs/guides/expectations/create_expectations_overview.md

Co-authored-by: talagluck <[email protected]>

* Changed "automatic" workflow to "Profiler" workflow.  Minor edits for grammar.

* changed "in depth" to "in-depth" where appropriate.

Co-authored-by: William Shin <[email protected]>
Co-authored-by: talagluck <[email protected]>
* Rough draft for "Validate Data: Overview" page

* Updated some phrasing for clarity.

* Update docs/guides/validation/validate_data_overview.md

Co-authored-by: talagluck <[email protected]>

Co-authored-by: talagluck <[email protected]>
# Conflicts:
#	docs/guides/connecting_to_your_data/index.md
#	src/css/custom.css
…s to text where the illustrations were inserted. (#4048)
… to text where the illustration was inserted. (#4053)
… edits to text where the illustration was inserted. (#4052)
* First Pass of creating glossary terms files.

* Added additional Technical Terms to the glossary, and added some notes for the eventual detailed definitions to some Technical Terms.

* Added docusaurus-terminology plugin and compiled the glossary page.

* reformatted and added the glossary.
removed links to details pages from glossary.md
Corrected typo in validation_result_store.md

* removed remaining links in glossary.md

* rough drafts/notes on datasource.md and store.md

* created _tag.mdx, which uses tag.json to easily provide Technical Terms with a hover text definition and url link.
Added custom styling necessary for hovertext
Corrected typo in batch_request.md
Tested _tag.mdx and terms.json imports in glossary.md (This will need to be redone, but for now it provides an example of how to use the _tag.mdx import in other files and should provide the basis for having glossary.md automatically update based on the contents of terms.json)

* Added all technical terms and definitions to terms.json

* Fixed typo in "Batch"

* Updates to sidebars.js to include technical terms folder.
Updates to technical term files to include tagged terms.
update _tag.mdx to accept relative path directions to the docs folder.

* Update terms.json

* Batch Definition Candidate Edits (#4020)

* Terms language tweaks

* First draft Batch extended definition

* datasource language tweak

* Added build_glossary_page.py as a utility script - it will generate an alphabetically sorted glossary page based on the contents of terms.json.

Updated the glossary.md using build_glossary_page.py

* Added build_glossary_page.py as a utility script - it will generate an alphabetically sorted glossary page based on the contents of terms.json.

Updated the glossary.md using build_glossary_page.py

* Minor edit to glossary of terms page for consistency in formatting (separating content from metadata by one line in source files.)

* Cleaned up custom CSS after conflict in import from parent branch.

Corrected links in batch.md

Co-authored-by: James Campbell <[email protected]>
* Consolidated all contributing documentation in the Contributing folder of the ToC

* Moved all contributing guides into the contributing folder.

Moved miscellaneous how to guides into appropriate folders based on when they are applicable.

Combined changelog and migrationg guide into "Updates and Migration" section.

Combined API Reference and Core Concepts into Reference section.

Moved how to use the GE CLI into Reference section.

Added Advanced section to Setup How to guides to contain moved miscellaneous guides.

* Updated definitions in terms.json and glossary to reduce emphasis on terms as objects.

* Added notice for GE Cloud into Setup: Overview document under "The alternative to manual Setup" section.

* Corrected "in depth guide" to "in-depth guide" (in-depth is the adjective version.)

* Added index page to "Deployment Guides" section; updated indexes in all "how to guide" sections.
* - Updated content for the Data Context technical term page.
- Minor edits to the Setup: Overview how-to guide index.

* Added tags for technical terms in data_context.md

* - Corrected some headings that were improperly title cased.

* - Updated admonition boxes to have proper syntax highlighting and titles.
* - Added Plugin technical term page and content.
- Added Plugin entry to terms.json

* - Updated Glossary to include Plugin entry.

* - minor edits for grammar/spelling

* Update docs/glossary.md

Co-authored-by: talagluck <[email protected]>

* Update plugin.md

Co-authored-by: talagluck <[email protected]>
* - Add Checkpoint technical term page content.
- Add Checkpoint technical term page to ToC.
- Add information from Checkpoint core concepts to other technical term page's notes sections (for when those documents are created in full.)

* - Corrected a technicla term that was not title cased.

* - Added technical term tags and updated technical terms to proper title case format.
* - Linked the Validation technical term to the "Validate Data: Overview" page.

* - Added reference to Plugins in setup_overview
- Minor correction to spelling in plugin doc.
* - Added base content to expectation_suite.md
- Added expectation_suite.md to the ToC

* - Added tags to Expectation Suite technical term page.
- Minor edits to Expectation Suite technical term page.

* Update docs/terms/expectation_suite.md

Co-authored-by: James Campbell <[email protected]>

* Update docs/terms/expectation_suite.md

Co-authored-by: James Campbell <[email protected]>

* Update docs/terms/expectation_suite.md

Co-authored-by: James Campbell <[email protected]>

* - updated Expectation Suite page Reusability section to account for individual Expectation Suites being paired with individual Batch Requests, rather than all Expectation Suites running on all Batch Requests in the Checkpoint.

Co-authored-by: James Campbell <[email protected]>
* Updated Validation Action to Action in the files and terms.json
Added Action content and page

* Added tags to technical terms.

* Corrected keys in some technical tags.

* - updated information on `data_asset` parameter in Actions to indicate it will be a Validator instance.

* Update docs/terms/action.md

Co-authored-by: talagluck <[email protected]>

* Update docs/terms/action.md

Co-authored-by: talagluck <[email protected]>

* Update docs/terms/action.md

Co-authored-by: talagluck <[email protected]>

* Update docs/terms/action.md

* - Updated Action definition to be more concise
- Added note regarding contributions and Slack consultation in relation to developing custom Actions.
- Corrected casing for Python
- Added clarification regarding order Actions are run in.

Co-authored-by: talagluck <[email protected]>
* - started Expectation documentation
- Added missing documentation to Execution Engine technical term page.

* - Expanded content in Expectation technical term page.

* - finished adding content to expectation.md

* - Added tags to expectation.md
- Converted Validation Action to Action in linked documents.

* -updates to sidebar layout; added Additional Documentation to the reference section, for docs that are supplemental to technical term documentation but aren't API documentation.

* slight tweak to Reference category in ToC

* - Added base content to expectation_suite.md
- Added expectation_suite.md to the ToC

* - Added tags to Expectation Suite technical term page.
- Minor edits to Expectation Suite technical term page.

* Update docs/terms/expectation_suite.md

Co-authored-by: James Campbell <[email protected]>

* Update docs/terms/expectation_suite.md

Co-authored-by: James Campbell <[email protected]>

* Update docs/terms/expectation_suite.md

Co-authored-by: James Campbell <[email protected]>

* Update docs/terms/execution_engine.md

Co-authored-by: talagluck <[email protected]>

* Update docs/terms/expectation.md

Co-authored-by: talagluck <[email protected]>

* Update docs/terms/expectation.md

Co-authored-by: talagluck <[email protected]>

* Update docs/terms/expectation.md

Co-authored-by: talagluck <[email protected]>

* Update docs/terms/expectation.md

Co-authored-by: talagluck <[email protected]>

* - Updates to the Expectation page based on conversations on GitHub.

* - minor edits to execution engine page.

* Update docs/terms/execution_engine.md

Co-authored-by: talagluck <[email protected]>

* - Added note pertaining to developing new Execution Engines.
- Removed section on Design motivations.

Co-authored-by: James Campbell <[email protected]>
Co-authored-by: talagluck <[email protected]>
- Added supporting resource technical term page to ToC
* - Updated definition of "Validation Result" to include `ExpectationValidationResult` and `ExpectationSuiteValidationResult` objects.
- Added content to "Validation Result" technical term page.
- Added "Validation Result" page to ToC.

* - Minor updates for clarity.

* - Additional edits for clarity between `ExpectationValidationResult` and `ExpectationSuiteValidationResult` instances.
- Added technical term tags.

* Update docs/terms/validation_result.md

Co-authored-by: talagluck <[email protected]>

* Update docs/terms/validation_result.md

Co-authored-by: talagluck <[email protected]>

* Update docs/terms/validation_result.md

Co-authored-by: talagluck <[email protected]>

* Update docs/terms/validation_result.md

Co-authored-by: talagluck <[email protected]>

* Update docs/terms/validation_result.md

Co-authored-by: talagluck <[email protected]>

* Update docs/terms/validation_result.md

Co-authored-by: talagluck <[email protected]>

* - added clarification of the differences between `ExpectationSuiteValidationResult` and `ExpectationValidationResult`.

Co-authored-by: talagluck <[email protected]>
* - Added content to "Profiler" technical term page.
- Added "Profiler" technical term page to the ToC

* - updated content to reference RuleBasedProfiler class instead of Profiler class in accordance with the updated how-to guides.

* Update docs/terms/profiler.md

Co-authored-by: talagluck <[email protected]>

* Update docs/terms/profiler.md

Co-authored-by: talagluck <[email protected]>

* - updates to the documentation based on GitHub discussions.

* - fixed missing comma in sidebars.js

Co-authored-by: talagluck <[email protected]>
* - Added content for Metric technical term page.
- Added Metric technical term page to ToC

* - Added technical term tags to Metric page
- Minor editing to comply with documentation standards
- Updated "Validation Action" references to "Action"

* - Added technical term tags to Metric page
- Minor editing to comply with documentation standards

* Update docs/terms/metric.md

Co-authored-by: talagluck <[email protected]>

* Update docs/terms/metric.md

Co-authored-by: talagluck <[email protected]>

* - updates to the documentation based on GitHub discussions.

* Removed nonsensical statement.  Will replace with clarified version if anyone can figure out what it was meant to convey.

* - Removed nonsensical statement.  Will replace with clarified version if anyone can figure out what it was meant to convey.
- Removed extraneous path information alongside github url.

* -Updated technical term tag for Action after merging in updates.

Co-authored-by: talagluck <[email protected]>
* - Added base content to technical term page for Data Docs.
- Added Data Docs technical term page to ToC.

* - Added technical term tags to the Data Docs technical term page.

* - Updated according to GitHub suggestions.
Rachel-Reverie and others added 17 commits February 25, 2022 11:42
* - Added initial content to Store technical term page.
- Added notes on interactions with other objects to other Store type technical term pages.
- Added Checkpoint Store to technical terms due to references in consolidated documentation.
- Added Store and Store subtype pages to ToC

* - Added initial content to Checkpoint, Expectation, and Validation Result Store pages.
- Added framework to Evaluation Parameter Store page.

* - Added Data Docs Stores to terms.json and Stores technical term page.
- Added framework for Data Docs Store technical term page.
- Added Data Docs Store technical term page to ToC

* - Added metric store page to ToC
- updated information regarding Metric Stores on Stores technical term page.

* - Added base content to technical term pages for the remaining subclasses of ValidationStore.

* - Updated links that Docusaurus wasn't parsing correctly.

* typo expectation_store.md

* typo validation_result_store.md

* - Made additions and corrections corresponding to GitHub conversation.

* - Added technical term tags to all Store pages.
- Minor edits.

Co-authored-by: Joshua Stauffer <[email protected]>
* - major overhaul of content for Data Asset technical term page (updating from when it was partially put together for a community meetup demo)

* - added technical term tags to Data Asset page.
Reference Architectures ToC content
… categories. Minor edits to category layouts.
Minor edits to terms.json
Removed glossary entry for Catalog Asset until an appropriate page/link can be added.
@netlify
Copy link

netlify bot commented Feb 28, 2022

✔️ Deploy Preview for niobium-lead-7998 ready!

🔨 Explore the source changes: d5d5536

🔍 Inspect the deploy log: https://app.netlify.com/sites/niobium-lead-7998/deploys/622621a189ecf20008f4670b

😎 Browse the preview: https://deploy-preview-4292--niobium-lead-7998.netlify.app

Rachel-Reverie and others added 6 commits February 28, 2022 10:59
# Conflicts:
#	docs/contributing/components/prerequisites.jsx
#	docs/glossary.md
#	docs/guides/expectations/creating_custom_expectations/how_to_create_custom_column_aggregate_expectations.md
#	docs/guides/expectations/creating_custom_expectations/how_to_create_custom_column_map_expectations.md
#	docs/guides/expectations/features_custom_expectations/how_to_add_example_cases_for_an_expectation.md
#	docs/guides/expectations/features_custom_expectations/how_to_add_input_validation_for_an_expectation.md
#	docs/guides/expectations/features_custom_expectations/how_to_add_spark_support_for_an_expectation.md
#	docs/guides/expectations/features_custom_expectations/how_to_add_sqlalchemy_support_for_an_expectation.md
#	examples/expectations/column_aggregate_expectation_template.py
#	examples/expectations/column_map_expectation_template.py
#	great_expectations/expectations/core/expect_column_values_to_be_between.py
#	great_expectations/expectations/core/expect_table_row_count_to_be_between.py
#	great_expectations/expectations/expectation.py
#	great_expectations/render/renderer/site_builder.py
#	great_expectations/rule_based_profiler/parameter_builder/metric_multi_batch_parameter_builder.py
#	great_expectations/rule_based_profiler/parameter_builder/numeric_metric_range_multi_batch_parameter_builder.py
#	great_expectations/rule_based_profiler/parameter_builder/parameter_builder.py
#	great_expectations/rule_based_profiler/parameter_builder/regex_pattern_string_parameter_builder.py
#	great_expectations/rule_based_profiler/parameter_builder/simple_date_format_string_parameter_builder.py
#	great_expectations/rule_based_profiler/types/__init__.py
#	great_expectations/rule_based_profiler/types/domain.py
#	great_expectations/rule_based_profiler/types/parameter_container.py
#	great_expectations/validator/validator.py
#	sidebars.js
#	tests/integration/docusaurus/expectations/creating_custom_expectations/expect_column_max_to_be_between_custom.py
#	tests/integration/profiling/rule_based_profilers/test_profiler_user_workflows.py
#	tests/integration/test_script_runner.py
Copy link
Member

@donaldheppner donaldheppner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Amazing work! It'll be great to get this merged and live!

@Rachel-Reverie Rachel-Reverie merged commit daa626b into develop Mar 7, 2022
@Rachel-Reverie Rachel-Reverie deleted the DOCS/DOC-9/DOC-159/universal_map_toc_update branch March 7, 2022 15:42
@fjork3 fjork3 changed the title Docs/doc 9/doc 159/universal map toc update [DOCS] Universal map TOC update Mar 10, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.