Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: Define a new makefile command for running tfplugindocs #1829

Merged
merged 8 commits into from
Jan 12, 2024

Conversation

andreaangiolillo
Copy link
Collaborator

@andreaangiolillo andreaangiolillo commented Jan 10, 2024

Description

Ticket: CLOUDP-221358

This PR defines the make generate-doc command to generate the resource tf doc.

Type of change:

  • Bug fix (non-breaking change which fixes an issue). Please, add the "bug" label to the PR.
  • New feature (non-breaking change which adds functionality). Please, add the "enhancement" label to the PR.
  • Breaking change (fix or feature that would cause existing functionality to not work as expected). Please, add the "breaking change" label to the PR.
  • This change requires a documentation update
  • Documentation fix/enhancement

Required Checklist:

  • I have signed the MongoDB CLA
  • I have read the contribution guidelines
  • I have checked that this change does not generate any credentials and that they are NOT accidentally logged anywhere.
  • I have added tests that prove my fix is effective or that my feature works per HashiCorp requirements
  • I have added any necessary documentation (if appropriate)
  • I have run make fmt and formatted my code
  • If changes include deprecations or removals, I defined an isolated PR with a relevant title as it will be used in the auto-generated changelog.

Testing

Running the new make command
export resource_name=search_deployment && make generate-doc                                                                                       
./scripts/generate-doc.sh resource_name=search_deployment
Warning: we coudn't find the template for the search_deployments data source
Please, make sure to include the data source template under templates/data-sources/search_deployment.html.markdown.tmpl
Skipping this check: We assume that the resource does not have a plural data source.

rendering website for provider "terraform-provider-mongodbatlas" (as "terraform-provider-mongodbatlas")
copying any existing content to tmp dir
exporting schema from Terraform
compiling provider "mongodbatlas"
downloading Terraform CLI binary version from releases.hashicorp.com: 1.6.6
running terraform init
getting provider schema
rendering missing docs
generating missing resource content
generating template for "mongodbatlas_organization"
generating template for "mongodbatlas_project_ip_access_list"
generating template for "mongodbatlas_ldap_verify"
generating template for "mongodbatlas_privatelink_endpoint_service"
generating template for "mongodbatlas_third_party_integration"
generating template for "mongodbatlas_project_invitation"
generating template for "mongodbatlas_cloud_backup_schedule"
generating template for "mongodbatlas_federated_settings_identity_provider"
generating template for "mongodbatlas_project_api_key"
generating template for "mongodbatlas_global_cluster_config"
generating template for "mongodbatlas_search_deployment"
generating template for "mongodbatlas_advanced_cluster"
generating template for "mongodbatlas_privatelink_endpoint_service_serverless"
generating template for "mongodbatlas_event_trigger"
generating template for "mongodbatlas_federated_database_instance"
generating template for "mongodbatlas_cluster"
generating template for "mongodbatlas_network_peering"
generating template for "mongodbatlas_privatelink_endpoint_service_data_federation_online_archive"
generating template for "mongodbatlas_cloud_backup_snapshot"
generating template for "mongodbatlas_encryption_at_rest"
generating template for "mongodbatlas_private_endpoint_regional_mode"
generating template for "mongodbatlas_maintenance_window"
generating template for "mongodbatlas_network_container"
generating template for "mongodbatlas_org_invitation"
generating template for "mongodbatlas_privatelink_endpoint"
generating template for "mongodbatlas_api_key"
generating template for "mongodbatlas_auditing"
generating template for "mongodbatlas_cluster_outage_simulation"
generating template for "mongodbatlas_online_archive"
generating template for "mongodbatlas_project"
generating template for "mongodbatlas_serverless_instance"
generating template for "mongodbatlas_x509_authentication_database_user"
generating template for "mongodbatlas_cloud_provider_access_setup"
generating template for "mongodbatlas_data_lake_pipeline"
generating template for "mongodbatlas_federated_settings_org_role_mapping"
generating template for "mongodbatlas_teams"
generating template for "mongodbatlas_access_list_api_key"
generating template for "mongodbatlas_custom_db_role"
generating template for "mongodbatlas_database_user"
generating template for "mongodbatlas_ldap_configuration"
generating template for "mongodbatlas_alert_configuration"
generating template for "mongodbatlas_backup_compliance_policy"
generating template for "mongodbatlas_cloud_provider_access_authorization"
generating template for "mongodbatlas_federated_query_limit"
generating template for "mongodbatlas_privatelink_endpoint_serverless"
generating template for "mongodbatlas_federated_settings_org_config"
generating template for "mongodbatlas_search_index"
generating template for "mongodbatlas_cloud_backup_snapshot_export_bucket"
generating template for "mongodbatlas_cloud_backup_snapshot_export_job"
generating template for "mongodbatlas_custom_dns_configuration_cluster_aws"
generating template for "mongodbatlas_cloud_backup_snapshot_restore_job"
generating template for "mongodbatlas_team"
generating missing data source content
generating template for "mongodbatlas_alert_configurations"
generating template for "mongodbatlas_cloud_backup_snapshot_export_job"
generating template for "mongodbatlas_federated_settings_identity_provider"
generating template for "mongodbatlas_shared_tier_snapshot"
generating template for "mongodbatlas_network_peerings"
generating template for "mongodbatlas_privatelink_endpoint"
generating template for "mongodbatlas_project_ip_access_list"
generating template for "mongodbatlas_search_index"
generating template for "mongodbatlas_advanced_clusters"
generating template for "mongodbatlas_cluster"
generating template for "mongodbatlas_data_lake_pipeline"
generating template for "mongodbatlas_event_trigger"
generating template for "mongodbatlas_x509_authentication_database_user"
generating template for "mongodbatlas_cloud_backup_snapshot"
generating template for "mongodbatlas_database_user"
generating template for "mongodbatlas_federated_settings_org_role_mappings"
generating template for "mongodbatlas_shared_tier_snapshots"
generating template for "mongodbatlas_roles_org_id"
generating template for "mongodbatlas_third_party_integration"
generating template for "mongodbatlas_cloud_provider_access_setup"
generating template for "mongodbatlas_data_lake_pipeline_runs"
generating template for "mongodbatlas_network_containers"
generating template for "mongodbatlas_privatelink_endpoint_service_data_federation_online_archive"
generating template for "mongodbatlas_cloud_backup_snapshot_export_bucket"
generating template for "mongodbatlas_organizations"
generating template for "mongodbatlas_project_api_keys"
generating template for "mongodbatlas_shared_tier_restore_jobs"
generating template for "mongodbatlas_search_indexes"
generating template for "mongodbatlas_access_list_api_keys"
generating template for "mongodbatlas_advanced_cluster"
generating template for "mongodbatlas_network_peering"
generating template for "mongodbatlas_project"
generating template for "mongodbatlas_serverless_instance"
generating template for "mongodbatlas_access_list_api_key"
generating template for "mongodbatlas_cloud_backup_snapshot_export_buckets"
generating template for "mongodbatlas_event_triggers"
generating template for "mongodbatlas_federated_query_limits"
generating template for "mongodbatlas_maintenance_window"
generating template for "mongodbatlas_organization"
generating template for "mongodbatlas_cloud_backup_snapshot_restore_job"
generating template for "mongodbatlas_clusters"
generating template for "mongodbatlas_federated_database_instance"
generating template for "mongodbatlas_ldap_configuration"
generating template for "mongodbatlas_alert_configuration"
generating template for "mongodbatlas_atlas_users"
generating template for "mongodbatlas_data_lake_pipelines"
generating template for "mongodbatlas_project_api_key"
generating template for "mongodbatlas_team"
generating template for "mongodbatlas_cluster_outage_simulation"
generating template for "mongodbatlas_federated_query_limit"
generating template for "mongodbatlas_federated_settings_org_role_mapping"
generating template for "mongodbatlas_private_endpoint_regional_mode"
generating template for "mongodbatlas_project_invitation"
generating template for "mongodbatlas_search_deployment"
generating template for "mongodbatlas_custom_db_role"
generating template for "mongodbatlas_custom_db_roles"
generating template for "mongodbatlas_database_users"
generating template for "mongodbatlas_global_cluster_config"
generating template for "mongodbatlas_auditing"
generating template for "mongodbatlas_cloud_backup_snapshots"
generating template for "mongodbatlas_online_archive"
generating template for "mongodbatlas_privatelink_endpoints_service_serverless"
generating template for "mongodbatlas_backup_compliance_policy"
generating template for "mongodbatlas_federated_settings_org_config"
generating template for "mongodbatlas_federated_database_instances"
generating template for "mongodbatlas_federated_settings"
generating template for "mongodbatlas_federated_settings_identity_providers"
generating template for "mongodbatlas_privatelink_endpoint_service_serverless"
generating template for "mongodbatlas_atlas_user"
generating template for "mongodbatlas_cloud_backup_schedule"
generating template for "mongodbatlas_custom_dns_configuration_cluster_aws"
generating template for "mongodbatlas_data_lake_pipeline_run"
generating template for "mongodbatlas_privatelink_endpoint_service_data_federation_online_archives"
generating template for "mongodbatlas_projects"
generating template for "mongodbatlas_serverless_instances"
generating template for "mongodbatlas_shared_tier_restore_job"
generating template for "mongodbatlas_api_key"
generating template for "mongodbatlas_cloud_backup_snapshot_restore_jobs"
generating template for "mongodbatlas_federated_settings_org_configs"
generating template for "mongodbatlas_online_archives"
generating template for "mongodbatlas_org_invitation"
generating template for "mongodbatlas_privatelink_endpoint_service"
generating template for "mongodbatlas_teams"
generating template for "mongodbatlas_third_party_integrations"
generating template for "mongodbatlas_api_keys"
generating template for "mongodbatlas_cloud_backup_snapshot_export_jobs"
generating template for "mongodbatlas_ldap_verify"
generating template for "mongodbatlas_network_container"
generating missing provider content
generating template for "terraform-provider-mongodbatlas"
rendering static website
cleaning rendered website dir
rendering templated website to static markdown
copying non-template file: ".DS_Store"
copying non-template file: "README.md"
rendering "data-sources/access_list_api_key.md.tmpl"
rendering "data-sources/access_list_api_keys.md.tmpl"
rendering "data-sources/advanced_cluster.md.tmpl"
rendering "data-sources/advanced_clusters.md.tmpl"
rendering "data-sources/alert_configuration.md.tmpl"
rendering "data-sources/alert_configurations.md.tmpl"
rendering "data-sources/api_key.md.tmpl"
rendering "data-sources/api_keys.md.tmpl"
rendering "data-sources/atlas_user.md.tmpl"
rendering "data-sources/atlas_users.md.tmpl"
rendering "data-sources/auditing.md.tmpl"
rendering "data-sources/backup_compliance_policy.md.tmpl"
rendering "data-sources/cloud_backup_schedule.md.tmpl"
rendering "data-sources/cloud_backup_snapshot.md.tmpl"
rendering "data-sources/cloud_backup_snapshot_export_bucket.md.tmpl"
rendering "data-sources/cloud_backup_snapshot_export_buckets.md.tmpl"
rendering "data-sources/cloud_backup_snapshot_export_job.md.tmpl"
rendering "data-sources/cloud_backup_snapshot_export_jobs.md.tmpl"
rendering "data-sources/cloud_backup_snapshot_restore_job.md.tmpl"
rendering "data-sources/cloud_backup_snapshot_restore_jobs.md.tmpl"
rendering "data-sources/cloud_backup_snapshots.md.tmpl"
rendering "data-sources/cloud_provider_access_setup.md.tmpl"
rendering "data-sources/cluster.md.tmpl"
rendering "data-sources/cluster_outage_simulation.md.tmpl"
rendering "data-sources/clusters.md.tmpl"
rendering "data-sources/custom_db_role.md.tmpl"
rendering "data-sources/custom_db_roles.md.tmpl"
rendering "data-sources/custom_dns_configuration_cluster_aws.md.tmpl"
rendering "data-sources/data_lake_pipeline.md.tmpl"
rendering "data-sources/data_lake_pipeline_run.md.tmpl"
rendering "data-sources/data_lake_pipeline_runs.md.tmpl"
rendering "data-sources/data_lake_pipelines.md.tmpl"
rendering "data-sources/database_user.md.tmpl"
rendering "data-sources/database_users.md.tmpl"
rendering "data-sources/event_trigger.md.tmpl"
rendering "data-sources/event_triggers.md.tmpl"
rendering "data-sources/federated_database_instance.md.tmpl"
rendering "data-sources/federated_database_instances.md.tmpl"
rendering "data-sources/federated_query_limit.md.tmpl"
rendering "data-sources/federated_query_limits.md.tmpl"
rendering "data-sources/federated_settings.md.tmpl"
rendering "data-sources/federated_settings_identity_provider.md.tmpl"
rendering "data-sources/federated_settings_identity_providers.md.tmpl"
rendering "data-sources/federated_settings_org_config.md.tmpl"
rendering "data-sources/federated_settings_org_configs.md.tmpl"
rendering "data-sources/federated_settings_org_role_mapping.md.tmpl"
rendering "data-sources/federated_settings_org_role_mappings.md.tmpl"
rendering "data-sources/global_cluster_config.md.tmpl"
rendering "data-sources/ldap_configuration.md.tmpl"
rendering "data-sources/ldap_verify.md.tmpl"
rendering "data-sources/maintenance_window.md.tmpl"
rendering "data-sources/network_container.md.tmpl"
rendering "data-sources/network_containers.md.tmpl"
rendering "data-sources/network_peering.md.tmpl"
rendering "data-sources/network_peerings.md.tmpl"
rendering "data-sources/online_archive.md.tmpl"
rendering "data-sources/online_archives.md.tmpl"
rendering "data-sources/org_invitation.md.tmpl"
rendering "data-sources/organization.md.tmpl"
rendering "data-sources/organizations.md.tmpl"
rendering "data-sources/private_endpoint_regional_mode.md.tmpl"
rendering "data-sources/privatelink_endpoint.md.tmpl"
rendering "data-sources/privatelink_endpoint_service.md.tmpl"
rendering "data-sources/privatelink_endpoint_service_data_federation_online_archive.md.tmpl"
rendering "data-sources/privatelink_endpoint_service_data_federation_online_archives.md.tmpl"
rendering "data-sources/privatelink_endpoint_service_serverless.md.tmpl"
rendering "data-sources/privatelink_endpoints_service_serverless.md.tmpl"
rendering "data-sources/project.md.tmpl"
rendering "data-sources/project_api_key.md.tmpl"
rendering "data-sources/project_api_keys.md.tmpl"
rendering "data-sources/project_invitation.md.tmpl"
rendering "data-sources/project_ip_access_list.md.tmpl"
rendering "data-sources/projects.md.tmpl"
rendering "data-sources/roles_org_id.md.tmpl"
rendering "data-sources/search_deployment.html.markdown.tmpl"
rendering "data-sources/search_deployment.md.tmpl"
rendering "data-sources/search_index.md.tmpl"
rendering "data-sources/search_indexes.md.tmpl"
rendering "data-sources/serverless_instance.md.tmpl"
rendering "data-sources/serverless_instances.md.tmpl"
rendering "data-sources/shared_tier_restore_job.md.tmpl"
rendering "data-sources/shared_tier_restore_jobs.md.tmpl"
rendering "data-sources/shared_tier_snapshot.md.tmpl"
rendering "data-sources/shared_tier_snapshots.md.tmpl"
rendering "data-sources/team.md.tmpl"
rendering "data-sources/teams.md.tmpl"
rendering "data-sources/third_party_integration.md.tmpl"
rendering "data-sources/third_party_integrations.md.tmpl"
rendering "data-sources/x509_authentication_database_user.md.tmpl"
rendering "index.md.tmpl"
rendering "resources/access_list_api_key.md.tmpl"
rendering "resources/advanced_cluster.md.tmpl"
rendering "resources/alert_configuration.md.tmpl"
rendering "resources/api_key.md.tmpl"
rendering "resources/auditing.md.tmpl"
rendering "resources/backup_compliance_policy.md.tmpl"
rendering "resources/cloud_backup_schedule.md.tmpl"
rendering "resources/cloud_backup_snapshot.md.tmpl"
rendering "resources/cloud_backup_snapshot_export_bucket.md.tmpl"
rendering "resources/cloud_backup_snapshot_export_job.md.tmpl"
rendering "resources/cloud_backup_snapshot_restore_job.md.tmpl"
rendering "resources/cloud_provider_access_authorization.md.tmpl"
rendering "resources/cloud_provider_access_setup.md.tmpl"
rendering "resources/cluster.md.tmpl"
rendering "resources/cluster_outage_simulation.md.tmpl"
rendering "resources/custom_db_role.md.tmpl"
rendering "resources/custom_dns_configuration_cluster_aws.md.tmpl"
rendering "resources/data_lake_pipeline.md.tmpl"
rendering "resources/database_user.md.tmpl"
rendering "resources/encryption_at_rest.md.tmpl"
rendering "resources/event_trigger.md.tmpl"
rendering "resources/federated_database_instance.md.tmpl"
rendering "resources/federated_query_limit.md.tmpl"
rendering "resources/federated_settings_identity_provider.md.tmpl"
rendering "resources/federated_settings_org_config.md.tmpl"
rendering "resources/federated_settings_org_role_mapping.md.tmpl"
rendering "resources/global_cluster_config.md.tmpl"
rendering "resources/ldap_configuration.md.tmpl"
rendering "resources/ldap_verify.md.tmpl"
rendering "resources/maintenance_window.md.tmpl"
rendering "resources/network_container.md.tmpl"
rendering "resources/network_peering.md.tmpl"
rendering "resources/online_archive.md.tmpl"
rendering "resources/org_invitation.md.tmpl"
rendering "resources/organization.md.tmpl"
rendering "resources/private_endpoint_regional_mode.md.tmpl"
rendering "resources/privatelink_endpoint.md.tmpl"
rendering "resources/privatelink_endpoint_serverless.md.tmpl"
rendering "resources/privatelink_endpoint_service.md.tmpl"
rendering "resources/privatelink_endpoint_service_data_federation_online_archive.md.tmpl"
rendering "resources/privatelink_endpoint_service_serverless.md.tmpl"
rendering "resources/project.md.tmpl"
rendering "resources/project_api_key.md.tmpl"
rendering "resources/project_invitation.md.tmpl"
rendering "resources/project_ip_access_list.md.tmpl"
rendering "resources/search_deployment.html.markdown.tmpl"
rendering "resources/search_deployment.md.tmpl"
rendering "resources/search_index.md.tmpl"
rendering "resources/serverless_instance.md.tmpl"
rendering "resources/team.md.tmpl"
rendering "resources/teams.md.tmpl"
rendering "resources/third_party_integration.md.tmpl"
rendering "resources/x509_authentication_database_user.md.tmpl"

Moving the generated file search_deployment.html.markdown to the website folder
Moving the generated file search_deployment.html.markdown to the website folderWarning: We cannot find the documentation file for the data source search_deployments.html.markdown.
Please, make sure to include the data source template under templates/data-sources/search_deployments.html.markdown.tmpl
Skipping this step: We assume that the resource does not have a plural data source.


The documentation for search_deployment has been created.


set -euo pipefail

TF_VERSION="${TF_VERSION:-"1.6.6"}" # TF version to use when running tfplugindocs. Default: 1.6.6
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think it'll be hard to remember to update this version but I understand we only have TF_VERSION_LATEST in GH not in local / makefile.

Can you try to see if it works with 1.6.x (for latest patch in 1.6) so we won't need to change it so often?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Curious if the terraform version has much impact in generated docs. Having a fixed version does help ensure consistent results.

Copy link
Member

@lantoli lantoli Jan 11, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that's a good point, i have no idea. on the other hand there can be interesting improvements that we miss if we don't try with latest versions

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Curious if the terraform version has much impact in generated docs. Having a fixed version does help ensure consistent results.

Unfortunately, their documentation does not explain how TF is used for the generation. I created hashicorp/terraform-plugin-docs#323 to get an answer from HashiCorp

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you try to see if it works with 1.6.x (for latest patch in 1.6) so we won't need to change it so often?

it did work indeed. Thanks! I will update the code

exporting schema from Terraform
compiling provider "mongodbatlas"
downloading Terraform CLI binary version from releases.hashicorp.com: 1.6

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated


resource_name="$1"

if [ ! -f "${TEMPLATE_FOLDER_PATH}/resources/${resource_name}.html.markdown.tmpl" ]; then
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

knitpit: to have only one if statement with the 3 paths in an array

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I prefer leaving the code as it is as the last if has a different behaviour and does not return an error if the template is missing since some resources do not have the plural form of the data source. Happy to create a follow-up PR if you have a strong opinion about this change. Thanks for the feedback!


tfplugindocs generate --tf-version "${TF_VERSION}" --website-source-dir "${TEMPLATE_FOLDER_PATH}"

if [ ! -f "docs/resources/${resource_name}.html.markdown" ]; then
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

knitpit: similar comment having only one if

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I prefer leaving the code as it is as the last if has a different behaviour and does not return an error if the template is missing since some resources do not have the plural form of the data source. Happy to create a follow-up PR if you have a strong opinion about this change. Thanks for the feedback!

@@ -121,3 +121,8 @@ scaffold:
@go run ./tools/scaffold/*.go $(name) $(type)
@echo "Reminder: configure the new $(type) in provider.go"


.PHONY: generate-doc
generate-doc: ## Generate the resource documentation via tfplugindocs
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this generated on demand? or we want to have something similar to test mocks to check if the user forgot to run the command by running it in a GH action and checking that there are not file changes.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, this can be useful to ensure docs and schema descriptions/changes are kept synced. This would however require an additional command that calls the docs generation for all resources which have documentation autogenerated.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that this is a good idea. I believe it is still early to implement it as it should require moving all the resources to the auto-generation of the doc so that we don't have to keep track of what resources are using the gen tool and which are not. I will create a ticket to track this work once all the resources are migrated

Copy link
Member

@AgustinBettati AgustinBettati left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@andreaangiolillo andreaangiolillo merged commit d1fba18 into master Jan 12, 2024
@andreaangiolillo andreaangiolillo deleted the CLOUDP-221358 branch January 12, 2024 11:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants