Adding new Lustre Resource #13181

stephanecharite · 2025-02-25T16:37:20Z

Creating a new terraform resource for Google Managed Lustre.

`google_lustre_instance`

modular-magician · 2025-02-25T16:48:26Z

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 13 files changed, 1490 insertions(+), 2 deletions(-))
google-beta provider: Diff ( 13 files changed, 1490 insertions(+), 2 deletions(-))
terraform-google-conversion: Diff ( 1 file changed, 128 insertions(+))
Open in Cloud Shell: Diff ( 4 files changed, 135 insertions(+))

Missing test report

Your PR includes resource fields which are not covered by any test.

Resource: google_lustre_instance (2 total tests)
Please add an acceptance test which includes these fields. The test should include the following:

resource "google_lustre_instance" "primary" {
  per_unit_storage_throughput = # value needed
}

modular-magician · 2025-02-25T18:46:14Z

Tests analytics

Total tests: 4612
Passed tests: 4176
Skipped tests: 432
Affected tests: 4

Click here to see the affected service packages

All service packages are affected

#### Non-exercised tests

🔴 Tests were added that are skipped in VCR:

TestAccLustreInstance_lustreInstanceBasicExample
TestAccLustreInstance_update

Action taken

Found 4 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

TestAccAccessContextManager

TestAccCloudbuildWorkerPool_basic

TestAccDataSourceGoogleGkeHubFeature_basic

TestAccEphemeralServiceAccountKey_basic

Get to know how VCR tests work

modular-magician · 2025-02-25T19:04:30Z

🟢 Tests passed during RECORDING mode:
TestAccAccessContextManager [Debug log]
TestAccDataSourceGoogleGkeHubFeature_basic [Debug log]

🔴 Tests failed when rerunning REPLAYING mode:
TestAccDataSourceGoogleGkeHubFeature_basic [Error message] [Debug log]

Tests failed due to non-determinism or randomness when the VCR replayed the response after the HTTP request was made.

Please fix these to complete your PR. If you believe these test failures to be incorrect or unrelated to your change, or if you have any questions, please raise the concern with your reviewer.

🔴 Tests failed during RECORDING mode:
TestAccCloudbuildWorkerPool_basic [Error message] [Debug log]
TestAccEphemeralServiceAccountKey_basic [Error message] [Debug log]

🔴 Errors occurred during RECORDING mode. Please fix them to complete your PR.

View the build log or the debug log for each test

modular-magician · 2025-02-25T19:47:51Z

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 13 files changed, 1490 insertions(+), 2 deletions(-))
google-beta provider: Diff ( 13 files changed, 1490 insertions(+), 2 deletions(-))
terraform-google-conversion: Diff ( 1 file changed, 128 insertions(+))
Open in Cloud Shell: Diff ( 4 files changed, 135 insertions(+))

Missing test report

Your PR includes resource fields which are not covered by any test.

Resource: google_lustre_instance (2 total tests)
Please add an acceptance test which includes these fields. The test should include the following:

resource "google_lustre_instance" "primary" {
  per_unit_storage_throughput = # value needed
}

modular-magician · 2025-02-25T21:47:30Z

Tests analytics

Total tests: 4627
Passed tests: 4189
Skipped tests: 434
Affected tests: 4

Click here to see the affected service packages

All service packages are affected

#### Non-exercised tests

🔴 Tests were added that are skipped in VCR:

TestAccLustreInstance_lustreInstanceBasicExample
TestAccLustreInstance_update

Action taken

Found 4 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

TestAccAccessContextManager

TestAccCloudbuildWorkerPool_basic

TestAccDataSourceGoogleGkeHubFeature_basic

TestAccEphemeralServiceAccountKey_basic

Get to know how VCR tests work

modular-magician · 2025-02-25T22:05:37Z

🟢 Tests passed during RECORDING mode:
TestAccAccessContextManager [Debug log]
TestAccDataSourceGoogleGkeHubFeature_basic [Debug log]

🔴 Tests failed when rerunning REPLAYING mode:
TestAccAccessContextManager [Error message] [Debug log]
TestAccDataSourceGoogleGkeHubFeature_basic [Error message] [Debug log]

Tests failed due to non-determinism or randomness when the VCR replayed the response after the HTTP request was made.

Please fix these to complete your PR. If you believe these test failures to be incorrect or unrelated to your change, or if you have any questions, please raise the concern with your reviewer.

🔴 Tests failed during RECORDING mode:
TestAccCloudbuildWorkerPool_basic [Error message] [Debug log]
TestAccEphemeralServiceAccountKey_basic [Error message] [Debug log]

🔴 Errors occurred during RECORDING mode. Please fix them to complete your PR.

View the build log or the debug log for each test

…guide

modular-magician · 2025-02-26T01:51:22Z

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 13 files changed, 1450 insertions(+), 2 deletions(-))
google-beta provider: Diff ( 13 files changed, 1450 insertions(+), 2 deletions(-))
terraform-google-conversion: Diff ( 1 file changed, 128 insertions(+))
Open in Cloud Shell: Diff ( 4 files changed, 135 insertions(+))

Missing test report

Your PR includes resource fields which are not covered by any test.

Resource: google_lustre_instance (2 total tests)
Please add an acceptance test which includes these fields. The test should include the following:

resource "google_lustre_instance" "primary" {
  per_unit_storage_throughput = # value needed
}

Missing service labels

The following new resources do not have corresponding service labels:

google_lustre_instance

If you believe this detection to be incorrect please raise the concern with your reviewer.
An override-missing-service-label label can be added to allow merging.

modular-magician · 2025-02-26T03:50:15Z

Tests analytics

Total tests: 4630
Passed tests: 4193
Skipped tests: 434
Affected tests: 3

Click here to see the affected service packages

All service packages are affected

#### Non-exercised tests

🔴 Tests were added that are skipped in VCR:

TestAccLustreInstance_lustreInstanceBasicExample
TestAccLustreInstance_update

Action taken

Found 3 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

TestAccCloudbuildWorkerPool_basic

TestAccDataSourceGoogleGkeHubFeature_basic

TestAccEphemeralServiceAccountKey_basic

Get to know how VCR tests work

modular-magician · 2025-02-26T03:54:42Z

🟢 Tests passed during RECORDING mode:
TestAccDataSourceGoogleGkeHubFeature_basic [Debug log]

🔴 Tests failed when rerunning REPLAYING mode:
TestAccDataSourceGoogleGkeHubFeature_basic [Error message] [Debug log]

Tests failed due to non-determinism or randomness when the VCR replayed the response after the HTTP request was made.

Please fix these to complete your PR. If you believe these test failures to be incorrect or unrelated to your change, or if you have any questions, please raise the concern with your reviewer.

🔴 Tests failed during RECORDING mode:
TestAccCloudbuildWorkerPool_basic [Error message] [Debug log]
TestAccEphemeralServiceAccountKey_basic [Error message] [Debug log]

🔴 Errors occurred during RECORDING mode. Please fix them to complete your PR.

View the build log or the debug log for each test

modular-magician · 2025-02-26T17:34:13Z

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 13 files changed, 1445 insertions(+), 2 deletions(-))
google-beta provider: Diff ( 13 files changed, 1445 insertions(+), 2 deletions(-))
terraform-google-conversion: Diff ( 1 file changed, 128 insertions(+))
Open in Cloud Shell: Diff ( 4 files changed, 135 insertions(+))

Missing test report

Your PR includes resource fields which are not covered by any test.

Resource: google_lustre_instance (2 total tests)
Please add an acceptance test which includes these fields. The test should include the following:

resource "google_lustre_instance" "primary" {
  per_unit_storage_throughput = # value needed
}

Missing service labels

The following new resources do not have corresponding service labels:

google_lustre_instance

If you believe this detection to be incorrect please raise the concern with your reviewer.
An override-missing-service-label label can be added to allow merging.

github-actions · 2025-02-26T17:48:18Z

Hello! I am a robot. Tests will require approval from a repository maintainer to run.

@roaks3, a repository maintainer, has been assigned to review your changes. If you have not received review feedback within 2 business days, please leave a comment on this PR asking them to take a look.

You can help make sure that review is quick by doing a self-review and by running impacted tests locally.

modular-magician · 2025-02-26T19:34:21Z

Tests analytics

Total tests: 4636
Passed tests: 4192
Skipped tests: 434
Affected tests: 10

Click here to see the affected service packages

All service packages are affected

Action taken

Found 10 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

TestAccAlloydbCluster_alloydbClusterAfterUpgradeExample

TestAccAlloydbCluster_alloydbClusterBeforeUpgradeExample

TestAccCloudbuildWorkerPool_basic

TestAccDataSourceGoogleGkeHubFeature_basic

TestAccEphemeralServiceAccountKey_basic

TestAccLustreInstance_lustreInstanceBasicExample

TestAccLustreInstance_update

TestAccSecretManagerSecretVersion_secretVersionBasicWriteOnlyExample

TestAccSecretManagerSecretVersion_secretVersionWithBase64StringSecretDataWriteOnlyExample

TestAccSqlUser_password_wo

Get to know how VCR tests work

modular-magician · 2025-02-26T19:55:34Z

🟢 Tests passed during RECORDING mode:
TestAccAlloydbCluster_alloydbClusterAfterUpgradeExample [Debug log]
TestAccAlloydbCluster_alloydbClusterBeforeUpgradeExample [Debug log]
TestAccDataSourceGoogleGkeHubFeature_basic [Debug log]

🔴 Tests failed when rerunning REPLAYING mode:
TestAccDataSourceGoogleGkeHubFeature_basic [Error message] [Debug log]

Tests failed due to non-determinism or randomness when the VCR replayed the response after the HTTP request was made.

Please fix these to complete your PR. If you believe these test failures to be incorrect or unrelated to your change, or if you have any questions, please raise the concern with your reviewer.

🔴 Tests failed during RECORDING mode:
TestAccCloudbuildWorkerPool_basic [Error message] [Debug log]
TestAccEphemeralServiceAccountKey_basic [Error message] [Debug log]
TestAccLustreInstance_lustreInstanceBasicExample [Error message] [Debug log]
TestAccLustreInstance_update [Error message] [Debug log]
TestAccSecretManagerSecretVersion_secretVersionBasicWriteOnlyExample [Error message] [Debug log]
TestAccSecretManagerSecretVersion_secretVersionWithBase64StringSecretDataWriteOnlyExample [Error message] [Debug log]
TestAccSqlUser_password_wo [Error message] [Debug log]

🔴 Errors occurred during RECORDING mode. Please fix them to complete your PR.

View the build log or the debug log for each test

modular-magician · 2025-02-27T18:00:25Z

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 13 files changed, 1445 insertions(+), 2 deletions(-))
google-beta provider: Diff ( 13 files changed, 1445 insertions(+), 2 deletions(-))
terraform-google-conversion: Diff ( 1 file changed, 128 insertions(+))
Open in Cloud Shell: Diff ( 4 files changed, 135 insertions(+))

Missing test report

Your PR includes resource fields which are not covered by any test.

Resource: google_lustre_instance (3 total tests)
Please add an acceptance test which includes these fields. The test should include the following:

resource "google_lustre_instance" "primary" {
  per_unit_storage_throughput = # value needed
}

Missing service labels

The following new resources do not have corresponding service labels:

google_lustre_instance

If you believe this detection to be incorrect please raise the concern with your reviewer.
An override-missing-service-label label can be added to allow merging.

modular-magician · 2025-02-27T20:00:04Z

Tests analytics

Total tests: 4641
Passed tests: 4198
Skipped tests: 434
Affected tests: 9

Click here to see the affected service packages

All service packages are affected

Action taken

Found 9 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

TestAccAccessContextManager

TestAccCloudbuildWorkerPool_basic

TestAccDataSourceGoogleGkeHubFeature_basic

TestAccEphemeralServiceAccountKey_basic

TestAccLustreInstance_lustreInstanceBasicExample

TestAccLustreInstance_update

TestAccSecretManagerSecretVersion_secretVersionBasicWriteOnlyExample

TestAccSecretManagerSecretVersion_secretVersionWithBase64StringSecretDataWriteOnlyExample

TestAccSqlUser_password_wo

Get to know how VCR tests work

modular-magician · 2025-02-27T20:19:11Z

🟢 Tests passed during RECORDING mode:
TestAccAccessContextManager [Debug log]
TestAccDataSourceGoogleGkeHubFeature_basic [Debug log]

🔴 Tests failed when rerunning REPLAYING mode:
TestAccAccessContextManager [Error message] [Debug log]
TestAccDataSourceGoogleGkeHubFeature_basic [Error message] [Debug log]

Tests failed due to non-determinism or randomness when the VCR replayed the response after the HTTP request was made.

Please fix these to complete your PR. If you believe these test failures to be incorrect or unrelated to your change, or if you have any questions, please raise the concern with your reviewer.

🔴 Tests failed during RECORDING mode:
TestAccCloudbuildWorkerPool_basic [Error message] [Debug log]
TestAccEphemeralServiceAccountKey_basic [Error message] [Debug log]
TestAccLustreInstance_lustreInstanceBasicExample [Error message] [Debug log]
TestAccLustreInstance_update [Error message] [Debug log]
TestAccSecretManagerSecretVersion_secretVersionBasicWriteOnlyExample [Error message] [Debug log]
TestAccSecretManagerSecretVersion_secretVersionWithBase64StringSecretDataWriteOnlyExample [Error message] [Debug log]
TestAccSqlUser_password_wo [Error message] [Debug log]

🔴 Errors occurred during RECORDING mode. Please fix them to complete your PR.

View the build log or the debug log for each test

roaks3 · 2025-02-28T21:14:27Z

/gcbrun

modular-magician · 2025-02-28T21:26:18Z

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 13 files changed, 1445 insertions(+), 2 deletions(-))
google-beta provider: Diff ( 13 files changed, 1445 insertions(+), 2 deletions(-))
terraform-google-conversion: Diff ( 1 file changed, 128 insertions(+))
Open in Cloud Shell: Diff ( 4 files changed, 135 insertions(+))

Missing test report

Your PR includes resource fields which are not covered by any test.

Resource: google_lustre_instance (3 total tests)
Please add an acceptance test which includes these fields. The test should include the following:

resource "google_lustre_instance" "primary" {
  per_unit_storage_throughput = # value needed
}

Missing service labels

The following new resources do not have corresponding service labels:

google_lustre_instance

If you believe this detection to be incorrect please raise the concern with your reviewer.
An override-missing-service-label label can be added to allow merging.

roaks3

Just a few comments on the initial pass. I've also enabled the API on our 3 test environments, so that should resolve the most recent test failures.

In addition to these changes, I believe you may also need to make changes to mmv1/third_party/terraform/.teamcity/components/inputs/services_beta.kt and mmv1/third_party/terraform/.teamcity/components/inputs/services_ga.kt to support nightly tests for the new product.

roaks3 · 2025-02-28T21:48:36Z

mmv1/products/lustre/Instance.yaml

+  - name: lustre_instance_basic
+    primary_resource_id: 'instance'
+    vars:
+      name: 'instance'


The ids here should each include a hyphen, which signals to our generator that we want the value used in tests to have a sweepable prefix and a randomize suffix. So for example, name could be my-instance or similar. I think the same applies to network_name and address_name.

roaks3 · 2025-02-28T21:48:39Z

mmv1/products/lustre/Instance.yaml

+      network_name: 'network'
+      address_name: 'ipaddress'
+      filesystem_name: 'filesystem'
+      capacity_gib: 21000


Values like this can be hardcoded (which it looks like you've already done actually)

roaks3 · 2025-02-28T21:48:49Z

mmv1/products/lustre/product.yaml

+versions:
+  - base_url: https://lustre.googleapis.com/v1/
+    name: ga
+caibaseurl: ""


This line can be removed for this resource (it's for products with urls like http://{location}-...)

roaks3 · 2025-02-28T21:48:56Z

mmv1/templates/terraform/examples/lustre_instance_basic.tf.tmpl

+resource "google_lustre_instance" "{{$.PrimaryResourceId}}" {
+  instance_id = "{{index $.Vars "name"}}"
+  location = "us-central1-a"
+  description = "test instance"


One of the test configs (here or in the update test) should have optional fields omitted, to verify they are in fact optional.

roaks3 · 2025-02-28T21:49:05Z

mmv1/third_party/terraform/services/lustre/resource_lustre_instance_test.go

+func testAccLustreInstance_full(context map[string]interface{}) string {
+	return acctest.Nprintf(`
+resource "google_lustre_instance" "instance" {
+  instance_id = "instance%{random_suffix}"


Similar to my other comment, the ids in the test should have a sweepable prefix ie. tf-test...

roaks3 · 2025-02-28T21:49:12Z

mmv1/third_party/terraform/services/lustre/resource_lustre_instance_test.go

+  location = "us-central1-a"
+  description = "test instance description field has been updated."
+  filesystem = "testfs"
+  capacity_gib = 14000


I was expecting to see more values changed here, like labels and capacity.

modular-magician · 2025-02-28T23:25:20Z

Tests analytics

Total tests: 4646
Passed tests: 4202
Skipped tests: 436
Affected tests: 8

Click here to see the affected service packages

All service packages are affected

Action taken

Found 8 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

TestAccCloudbuildWorkerPool_basic

TestAccDataSourceGoogleGkeHubFeature_basic

TestAccEphemeralServiceAccountKey_basic

TestAccLustreInstance_lustreInstanceBasicExample

TestAccLustreInstance_update

TestAccSecretManagerSecretVersion_secretVersionBasicWriteOnlyExample

TestAccSecretManagerSecretVersion_secretVersionWithBase64StringSecretDataWriteOnlyExample

TestAccSqlUser_password_wo

Get to know how VCR tests work

modular-magician · 2025-02-28T23:37:33Z

🟢 Tests passed during RECORDING mode:
TestAccDataSourceGoogleGkeHubFeature_basic [Debug log]

🔴 Tests failed when rerunning REPLAYING mode:
TestAccDataSourceGoogleGkeHubFeature_basic [Error message] [Debug log]

Tests failed due to non-determinism or randomness when the VCR replayed the response after the HTTP request was made.

Please fix these to complete your PR. If you believe these test failures to be incorrect or unrelated to your change, or if you have any questions, please raise the concern with your reviewer.

🔴 Tests failed during RECORDING mode:
TestAccCloudbuildWorkerPool_basic [Error message] [Debug log]
TestAccEphemeralServiceAccountKey_basic [Error message] [Debug log]
TestAccLustreInstance_lustreInstanceBasicExample [Error message] [Debug log]
TestAccLustreInstance_update [Error message] [Debug log]
TestAccSecretManagerSecretVersion_secretVersionBasicWriteOnlyExample [Error message] [Debug log]
TestAccSecretManagerSecretVersion_secretVersionWithBase64StringSecretDataWriteOnlyExample [Error message] [Debug log]
TestAccSqlUser_password_wo [Error message] [Debug log]

🔴 Errors occurred during RECORDING mode. Please fix them to complete your PR.

View the build log or the debug log for each test

stephanecharite added 3 commits February 21, 2025 15:35

autogenerating lustre terraform configurations

f328696

fixed integration tests

87f3999

resolving merge conflict

3fd8c57

addressed issue with linter

e9f9eaf

removing testAccCheckLustreInstanceDestroyProducer as per onboarding …

07e616b

…guide

fixed linting errors with the go test

df56734

stephanecharite marked this pull request as ready for review February 26, 2025 17:47

github-actions bot requested a review from roaks3 February 26, 2025 17:48

addressed copy paste errors while generating provider

cf11b13

roaks3 reviewed Feb 28, 2025

View reviewed changes

roaks3 mentioned this pull request Mar 3, 2025

Enable lustre service for test environments #13242

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adding new Lustre Resource #13181

Adding new Lustre Resource #13181

stephanecharite commented Feb 25, 2025

modular-magician commented Feb 25, 2025

modular-magician commented Feb 25, 2025

modular-magician commented Feb 25, 2025

modular-magician commented Feb 25, 2025

modular-magician commented Feb 25, 2025

modular-magician commented Feb 25, 2025

modular-magician commented Feb 26, 2025

modular-magician commented Feb 26, 2025

modular-magician commented Feb 26, 2025

modular-magician commented Feb 26, 2025

github-actions bot commented Feb 26, 2025

modular-magician commented Feb 26, 2025

modular-magician commented Feb 26, 2025

modular-magician commented Feb 27, 2025

modular-magician commented Feb 27, 2025

modular-magician commented Feb 27, 2025

roaks3 commented Feb 28, 2025

modular-magician commented Feb 28, 2025

roaks3 left a comment

roaks3 Feb 28, 2025

roaks3 Feb 28, 2025

roaks3 Feb 28, 2025

roaks3 Feb 28, 2025

roaks3 Feb 28, 2025

roaks3 Feb 28, 2025

modular-magician commented Feb 28, 2025

modular-magician commented Feb 28, 2025

Adding new Lustre Resource #13181

Are you sure you want to change the base?

Adding new Lustre Resource #13181

Conversation

stephanecharite commented Feb 25, 2025

modular-magician commented Feb 25, 2025

Diff report

Missing test report

modular-magician commented Feb 25, 2025

Tests analytics

Action taken

modular-magician commented Feb 25, 2025

modular-magician commented Feb 25, 2025

Diff report

Missing test report

modular-magician commented Feb 25, 2025

Tests analytics

Action taken

modular-magician commented Feb 25, 2025

modular-magician commented Feb 26, 2025

Diff report

Missing test report

Missing service labels

modular-magician commented Feb 26, 2025

Tests analytics

Action taken

modular-magician commented Feb 26, 2025

modular-magician commented Feb 26, 2025

Diff report

Missing test report

Missing service labels

github-actions bot commented Feb 26, 2025

modular-magician commented Feb 26, 2025

Tests analytics

Action taken

modular-magician commented Feb 26, 2025

modular-magician commented Feb 27, 2025

Diff report

Missing test report

Missing service labels

modular-magician commented Feb 27, 2025

Tests analytics

Action taken

modular-magician commented Feb 27, 2025

roaks3 commented Feb 28, 2025

modular-magician commented Feb 28, 2025

Diff report

Missing test report

Missing service labels

roaks3 left a comment

Choose a reason for hiding this comment

roaks3 Feb 28, 2025

Choose a reason for hiding this comment

roaks3 Feb 28, 2025

Choose a reason for hiding this comment

roaks3 Feb 28, 2025

Choose a reason for hiding this comment

roaks3 Feb 28, 2025

Choose a reason for hiding this comment

roaks3 Feb 28, 2025

Choose a reason for hiding this comment

roaks3 Feb 28, 2025

Choose a reason for hiding this comment

modular-magician commented Feb 28, 2025

Tests analytics

Action taken

modular-magician commented Feb 28, 2025