Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add data source for apphub discovered service #10105

Merged
merged 24 commits into from
Mar 12, 2024
Merged
Show file tree
Hide file tree
Changes from 10 commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
ddf5857
Add data source for apphub discovered service
gurankitt Mar 4, 2024
67b35df
Add data source for apphub discovered service'
gurankitt Mar 4, 2024
c1d55e9
Add data source for apphub discovered service
gurankitt Mar 4, 2024
a9a8103
resolved comments
gurankitt Mar 6, 2024
f3f5e02
retry logic added
gurankitt Mar 6, 2024
361919f
add tests and documentation
gurankitt Mar 7, 2024
bcaf67d
Corrected tests and added project field in the data source
gurankitt Mar 8, 2024
a8d409f
changed id field
gurankitt Mar 8, 2024
607e5df
Added random_suffix for the resources created and enabled iam policy …
gurankitt Mar 8, 2024
671b017
modified retry logic
gurankitt Mar 8, 2024
9284d56
Modified schema for the data source, and tests
gurankitt Mar 8, 2024
4239f4f
Removed IAM permission blocks
gurankitt Mar 8, 2024
9c075da
Add dependency for compute api
gurankitt Mar 8, 2024
b61d3e5
Merge branch 'GoogleCloudPlatform:main' into datasource
gurankitt Mar 8, 2024
86b1a6b
Resolve merge conflict
gurankitt Mar 8, 2024
9f2d90e
Add time sleep
gurankitt Mar 11, 2024
6d0ca32
Modified test function name
gurankitt Mar 11, 2024
7102010
Shorten service project name
gurankitt Mar 11, 2024
e4c5f15
Add billing account
gurankitt Mar 11, 2024
1325323
corrected get env variable function call
gurankitt Mar 11, 2024
afbdf9a
Modified project id
gurankitt Mar 11, 2024
a2a9cd1
Combined time delay
gurankitt Mar 11, 2024
9c7c92e
Resolving comments
gurankitt Mar 11, 2024
a5d541b
Remove retry logic and add time sleep for resource ingestion
gurankitt Mar 12, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ import (
"github.com/hashicorp/terraform-provider-google/google/services/dataflow"
"github.com/hashicorp/terraform-provider-google/google/services/servicenetworking"
"github.com/hashicorp/terraform-provider-google/google/tpgiamresource"
"github.com/hashicorp/terraform-provider-google/google/services/apphub"
)

// Datasources
Expand All @@ -28,6 +29,7 @@ var handwrittenDatasources = map[string]*schema.Resource{
"google_alloydb_supported_database_flags": alloydb.DataSourceAlloydbSupportedDatabaseFlags(),
"google_artifact_registry_repository": artifactregistry.DataSourceArtifactRegistryRepository(),
"google_app_engine_default_service_account": appengine.DataSourceGoogleAppEngineDefaultServiceAccount(),
"google_apphub_discovered_service": apphub.DataSourceApphubDiscoveredService(),
<% unless version == 'ga' -%>
"google_backup_dr_management_server": backupdr.DataSourceGoogleCloudBackupDRService(),
<% end -%>
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,210 @@
package apphub

import (
"fmt"
"log"

"github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema"
"github.com/hashicorp/terraform-provider-google/google/tpgresource"
transport_tpg "github.com/hashicorp/terraform-provider-google/google/transport"
"google.golang.org/api/googleapi"
)

func DataSourceApphubDiscoveredService() *schema.Resource {
roaks3 marked this conversation as resolved.
Show resolved Hide resolved
return &schema.Resource{
Read: dataSourceApphubDiscoveredServiceRead,
Schema: map[string]*schema.Schema{
"project": {
Type: schema.TypeString,
Optional: true,
},
"location": {
Type: schema.TypeString,
Required: true,
},
"service_uri": {
Type: schema.TypeString,
Required: true,
},
"discovered_service": {
roaks3 marked this conversation as resolved.
Show resolved Hide resolved
Type: schema.TypeList,
Computed: true,
Elem: &schema.Resource{
Schema: map[string]*schema.Schema{
"name": {
Type: schema.TypeString,
Computed: true,
},
"service_reference": {
Type: schema.TypeList,
Computed: true,
Elem: &schema.Resource{
Schema: map[string]*schema.Schema{
"uri": {
Type: schema.TypeString,
Computed: true,
},
"path": {
Type: schema.TypeString,
Computed: true,
},
},
},
},
"service_properties": {
Type: schema.TypeList,
Computed: true,
Elem: &schema.Resource{
Schema: map[string]*schema.Schema{
"gcp_project": {
Type: schema.TypeString,
Computed: true,
},
"location": {
Type: schema.TypeString,
Computed: true,
},
"zone": {
Type: schema.TypeString,
Computed: true,
},
},
},
},
},
},
},
},
}
}

func dataSourceApphubDiscoveredServiceRead(d *schema.ResourceData, meta interface{}) error {
config := meta.(*transport_tpg.Config)
userAgent, err := tpgresource.GenerateUserAgentString(d, config.UserAgent)
if err != nil {
return err
}

url, err := tpgresource.ReplaceVars(d, config, "{{ApphubBasePath}}projects/{{project}}/locations/{{location}}/discoveredServices:lookup?uri={{service_uri}}")
if err != nil {
return err
}

billingProject := ""

// err == nil indicates that the billing_project value was found
if bp, err := tpgresource.GetBillingProject(d, config); err == nil {
billingProject = bp
}

var res map[string]interface{}

err = transport_tpg.Retry(transport_tpg.RetryOptions{
RetryFunc: func() (rerr error) {
res, rerr = transport_tpg.SendRequest(transport_tpg.SendRequestOptions{
Config: config,
Method: "GET",
Project: billingProject,
RawURL: url,
UserAgent: userAgent,
})
return rerr
},
Timeout: d.Timeout(schema.TimeoutRead),
ErrorRetryPredicates: []transport_tpg.RetryErrorPredicateFunc{
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm trying to understand where this retry behavior is coming from, and why you need it. First, I believe IsNotFoundRetryableError will do the same thing, but also, the user could very easily configure a data source for a resource that doesn't exist, and end up waiting the full timeout to find that out. What case are we considering here?

func(err error) (bool, string) {
gerr, ok := err.(*googleapi.Error)
if !ok {
return false, ""
}

if gerr.Code == 404 {
log.Printf("[DEBUG] Dismissed an error as retryable based on error code: %s", err)
return true, fmt.Sprintf("Retryable error code %d", gerr.Code)
}
return false, ""
},
},
})

if err != nil {
return transport_tpg.HandleDataSourceNotFoundError(err, d, fmt.Sprintf("ApphubDiscoveredService %q", d.Id()), url)
}

if err := d.Set("discovered_service", flattenApphubDiscoveredService(res["discoveredService"], d, config)); err != nil {
return fmt.Errorf("Error setting discovered service: %s", err)
}

d.SetId(res["discoveredService"].(map[string]interface{})["name"].(string))

return nil

}

func flattenApphubDiscoveredService(v interface{}, d *schema.ResourceData, config *transport_tpg.Config) interface{} {
if v == nil {
return nil
}
original := v.(map[string]interface{})
if len(original) == 0 {
return nil
}
transformed := make(map[string]interface{})
transformed["name"] = flattenApphubDiscoveredServiceDataName(original["name"], d, config)
transformed["service_reference"] = flattenApphubServiceReference(original["serviceReference"], d, config)
transformed["service_properties"] = flattenApphubServiceProperties(original["serviceProperties"], d, config)
return []interface{}{transformed}
}

func flattenApphubServiceReference(v interface{}, d *schema.ResourceData, config *transport_tpg.Config) interface{} {
if v == nil {
return nil
}
original := v.(map[string]interface{})
if len(original) == 0 {
return nil
}
transformed := make(map[string]interface{})
transformed["uri"] = flattenApphubDiscoveredServiceDataUri(original["uri"], d, config)
transformed["path"] = flattenApphubDiscoveredServiceDataPath(original["path"], d, config)
return []interface{}{transformed}
}

func flattenApphubServiceProperties(v interface{}, d *schema.ResourceData, config *transport_tpg.Config) interface{} {
if v == nil {
return nil
}
original := v.(map[string]interface{})
if len(original) == 0 {
return nil
}
transformed := make(map[string]interface{})
transformed["gcp_project"] = flattenApphubDiscoveredServiceDataGcpProject(original["gcpProject"], d, config)
transformed["location"] = flattenApphubDiscoveredServiceDataLocation(original["location"], d, config)
transformed["zone"] = flattenApphubDiscoveredServiceDataZone(original["zone"], d, config)
return []interface{}{transformed}
}

func flattenApphubDiscoveredServiceDataName(v interface{}, d *schema.ResourceData, config *transport_tpg.Config) interface{} {
return v
}

func flattenApphubDiscoveredServiceDataUri(v interface{}, d *schema.ResourceData, config *transport_tpg.Config) interface{} {
return v
}

func flattenApphubDiscoveredServiceDataPath(v interface{}, d *schema.ResourceData, config *transport_tpg.Config) interface{} {
return v
}

func flattenApphubDiscoveredServiceDataGcpProject(v interface{}, d *schema.ResourceData, config *transport_tpg.Config) interface{} {
return v
}

func flattenApphubDiscoveredServiceDataLocation(v interface{}, d *schema.ResourceData, config *transport_tpg.Config) interface{} {
return v
}

func flattenApphubDiscoveredServiceDataZone(v interface{}, d *schema.ResourceData, config *transport_tpg.Config) interface{} {
return v
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
package apphub_test

import (
"testing"

"github.com/hashicorp/terraform-plugin-sdk/v2/helper/resource"
"github.com/hashicorp/terraform-provider-google/google/acctest"
)

func TestDataSourceApphubDiscoveredService_basic(t *testing.T) {
t.Parallel()

context := map[string]interface{}{
"random_suffix": acctest.RandString(t, 10),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we append random_suffix to resources created in the tests - to allow for parallel test runs?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

}

acctest.VcrTest(t, resource.TestCase{
PreCheck: func() { acctest.AccTestPreCheck(t) },
ProtoV5ProviderFactories: acctest.ProtoV5ProviderFactories(t),
Steps: []resource.TestStep{
{
Config: testDataSourceApphubDiscoveredService_basic(context),
},
},
})
}

func testDataSourceApphubDiscoveredService_basic(context map[string]interface{}) string {
return acctest.Nprintf(
`
resource "google_project" "service_project" {
project_id ="<%= ctx[:vars]['service_project_attachment_id'] %>-%{random_suffix}"
roaks3 marked this conversation as resolved.
Show resolved Hide resolved
name = "Service Project"
org_id = "<%= ctx[:test_env_vars]['org_id'] %>"
roaks3 marked this conversation as resolved.
Show resolved Hide resolved
}

resource "google_project_iam_binding" "apphub_service_project_iam_binding" {
roaks3 marked this conversation as resolved.
Show resolved Hide resolved
project = google_project.service_project.project_id
role = "roles/apphub.admin"
members = [
"<%= 'serviceAccount:'+ ctx[:test_env_vars]['service_account'] %>",
]
}

# Enable Compute API
roaks3 marked this conversation as resolved.
Show resolved Hide resolved
resource "google_project_service" "compute_service_project" {
project_id = google_project.service_project.project_id
service = "compute.googleapis.com"
}

resource "google_apphub_service_project_attachment" "service_project_attachment" {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should also add a iam-policy-binding; to be able to perform this attachment

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

service_project_attachment_id = google_project.service_project.project_id
}

# discovered service block
data "google_apphub_discovered_service" "catalog-service" {
provider = google
location = "us-east1"
roaks3 marked this conversation as resolved.
Show resolved Hide resolved
# ServiceReference | Application Hub | Google Cloud
# Using this reference means that this resource will not be provisioned until the forwarding rule is fully created
service_uri = "//compute.googleapis.com/${google_compute_forwarding_rule.forwarding_rule.id}"
depends_on = [google_apphub_service_project_attachment.service_project_attachment]
}

# VPC network
resource "google_compute_network" "ilb_network" {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have we enabled compute API for service_project?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

name = "<%= ctx[:vars]['ilb_network'] %>-%{random_suffix}"
project = google_project.service_project.project_id
auto_create_subnetworks = false
}

# backend subnet
resource "google_compute_subnetwork" "ilb_subnet" {
name = "<%= ctx[:vars]['ilb_subnet'] %>-%{random_suffix}"
project = google_project.service_project.project_id
ip_cidr_range = "10.0.1.0/24"
region = "us-east1"
network = google_compute_network.ilb_network.id
}

# forwarding rule
resource "google_compute_forwarding_rule" "forwarding_rule" {
name ="<%= ctx[:vars]['forwarding_rule'] %>-%{random_suffix}"
project = google_project.service_project.project_id
region = "us-east1"
ip_version = "IPV4"
load_balancing_scheme = "INTERNAL"
all_ports = true
backend_service = google_compute_region_backend_service.backend.id
network = google_compute_network.ilb_network.id
subnetwork = google_compute_subnetwork.ilb_subnet.id
}

# backend service
resource "google_compute_region_backend_service" "backend" {
name = "<%= ctx[:vars]['backend_service'] %>-%{random_suffix}"
project = google_project.service_project.project_id
region = "us-east1"
health_checks = [google_compute_health_check.default.id]
}

# health check
resource "google_compute_health_check" "default" {
name = "<%= ctx[:vars]['health_check'] %>-%{random_suffix}"
project = google_project.service_project.project_id
check_interval_sec = 1
timeout_sec = 1

tcp_health_check {
port = "80"
}
}
`, context)
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
---
subcategory: "Apphub"
description: |-
Get information about a discovered service.
---

# google\_apphub\_discovered_service

Get information about a discovered service from its uri.


## Example Usage


```hcl
data "google_apphub_discovered_service" "my-service" {
location = "my-location"
service_uri = "my-service-uri"
}
```

## Argument Reference

The following arguments are supported:

* `project` - The host project of the discovered service.
* `service_uri` - (Required) The uri of the service.
* `location` - (Required) The location of the discovered service.

## Attributes Reference

In addition to the arguments listed above, the following computed attributes are exported:

* `discovered_service` - Represents a network/api interface that exposes some functionality to clients for consumption over the network. A `discovered_service` object would contain the following fields:-

- `name` - Resource name of a Service. Format: "projects/{host-project-id}/locations/{location}/applications/{application-id}/services/{service-id}".

- `service_reference` - Reference to an underlying networking resource that can comprise a Service. A `service_reference` object would contain the following fields:-
- uri - The underlying resource URI.
- path - Additional path under the resource URI.

- `service_properties` - Properties of an underlying compute resource that can comprise a Service. A `service_properties` object would contain the following fields:-
- gcp_project - The service project identifier that the underlying cloud resource resides in.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These will be displayed on the website, so I'm not sure if the indentation changes how it renders, but we will want these bullets to be consistent (for example, between service_reference and service_properties).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

- location - The location that the underlying resource resides in.
- zone - The location that the underlying resource resides in if it is zonal.