Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add PDC network and PDC network token resources #1975

Merged
merged 22 commits into from
Jan 20, 2025
Merged

Conversation

aangelisc
Copy link
Contributor

@aangelisc aangelisc commented Jan 3, 2025

Feedback welcome here (my first time creating some TF resources), this is essentially a duplicate of the access policy and token resources with some minor changes that are specific to PDC.

Fixes #1222
Fixes #1638
Fixes https://github.com/grafana/hosted-grafana/issues/4278

- Add resource lister function that will return a slice of resources
@aangelisc aangelisc self-assigned this Jan 3, 2025
@aangelisc aangelisc requested review from a team as code owners January 3, 2025 17:39
Copy link

github-actions bot commented Jan 3, 2025

In order to lower resource usage and have a faster runtime, PRs will not run Cloud tests automatically.
To do so, a Grafana Labs employee must trigger the cloud acceptance tests workflow manually.

@aangelisc aangelisc requested review from julienduchesne and dafydd-t and removed request for a team January 3, 2025 19:15
Copy link

@dafydd-t dafydd-t left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks really good! Thanks a lot for starting it off. I've added some comments around the naming of things for now. We need a more thorough review of the terraform itself, we can get to that soon.

@aangelisc
Copy link
Contributor Author

Updated based on your suggestions @dafydd-t. I've also added a data source to allow retrieval of networks that may not be managed by TF.

@aangelisc aangelisc requested a review from dafydd-t January 6, 2025 21:02
@dafydd-t
Copy link

dafydd-t commented Jan 8, 2025

@aangelisc do you have an example terraform plan that you used to run this locally? I'd love to give it a go

@aangelisc
Copy link
Contributor Author

Hey @dafydd-t, I've pushed some more updates 😊

You should be able to test this with the following tf:

terraform {
  required_providers {
    grafana = {
      source = "grafana/grafana"
    }
  }
}

provider "grafana" {
  alias                     = "cloud"
  cloud_access_policy_token = "<token>"
}


data "grafana_cloud_stack" "stack" {
  provider = grafana.cloud

  slug = "stack"
}

resource "grafana_cloud_stack_service_account" "cloud_sa" {
  provider   = grafana.cloud
  stack_slug = data.grafana_cloud_stack.stack.slug

  name        = "pdc-test-sa"
  role        = "Admin"
  is_disabled = false
}

resource "grafana_cloud_stack_service_account_token" "cloud_sa" {
  provider   = grafana.cloud
  stack_slug = data.grafana_cloud_stack.stack.slug

  name               = "terraform serviceaccount key"
  service_account_id = grafana_cloud_stack_service_account.cloud_sa.id
}

provider "grafana" {
  alias = "stack"

  url                       = data.grafana_cloud_stack.stack.url
  auth                      = grafana_cloud_stack_service_account_token.cloud_sa.key
  cloud_access_policy_token = "<token>"
}

resource "grafana_cloud_private_data_source_connect_network" "test" {
  provider = grafana.stack

  region           = data.grafana_cloud_stack.stack.region_slug
  name             = "test-pdc"
  display_name     = "Test PDC"
  stack_identifier = data.grafana_cloud_stack.stack.id
}

resource "grafana_cloud_private_data_source_connect_network_token" "test" {
  provider = grafana.stack

  pdc_network_id = grafana_cloud_private_data_source_connect_network.test.pdc_network_id
  region         = grafana_cloud_private_data_source_connect_network.test.region
  name           = "my-pdc-token"
  display_name   = "My PDC Token"
}

resource "grafana_data_source" "influxdb" {
  provider = grafana.stack

  type                = "influxdb"
  name                = "pdc-test-influx"
  url                 = "http://influxdb.example.net:8086/"
  basic_auth_enabled  = true
  basic_auth_username = "username"
  database_name       = "dbname" // Example: influxdb_database.metrics.name

  json_data_encoded = jsonencode({
    authType          = "default"
    basicAuthPassword = "mypassword"
  })

  private_data_source_connect_network_id = grafana_cloud_private_data_source_connect_network.test.pdc_network_id
}

@dafydd-t
Copy link

Thanks - I tried this myself to create a cloud stack, PDC network and some tokens and it works a treat!

One behaviour i don't understand is when I run terraform apply with no changes, I see this:

# grafana_data_source.influxdb will be updated in-place
  ~ resource "grafana_data_source" "influxdb" {
        id                                     = "1:be9kz8qdpwbnka"
      ~ json_data_encoded                      = jsonencode(
          ~ {
              - enableSecureSocksProxy   = true
              - secureSocksProxyUsername = "99d012ce-4db8-4810-887f-90479122d333"
                # (2 unchanged attributes hidden)
            }
        )
        name                                   = "pdc-test-influx"
        # (11 unchanged attributes hidden)
    }

Why are these properties in the datasource getting deleted? But also, when i apply it and go check the datasource, it still looks configured correctly :/

Apart from that, from a PDC perspective this looks good! I've asked again in #platform for a terraform focused review.

@aangelisc
Copy link
Contributor Author

Thanks - I tried this myself to create a cloud stack, PDC network and some tokens and it works a treat!

One behaviour i don't understand is when I run terraform apply with no changes, I see this:

# grafana_data_source.influxdb will be updated in-place
  ~ resource "grafana_data_source" "influxdb" {
        id                                     = "1:be9kz8qdpwbnka"
      ~ json_data_encoded                      = jsonencode(
          ~ {
              - enableSecureSocksProxy   = true
              - secureSocksProxyUsername = "99d012ce-4db8-4810-887f-90479122d333"
                # (2 unchanged attributes hidden)
            }
        )
        name                                   = "pdc-test-influx"
        # (11 unchanged attributes hidden)
    }

Why are these properties in the datasource getting deleted? But also, when i apply it and go check the datasource, it still looks configured correctly :/

Apart from that, from a PDC perspective this looks good! I've asked again in #platform for a terraform focused review.

I've fixed this! It's because the data source jsonData value for the diff is determined before the actual final value is calculated. I've updated this to suppress the diff if the private_data_source_connect_network_id is set by also setting the relevant jsonData properties.

@dafydd-t
Copy link

That issue looks fixed! Thanks. I'm still seeing one unexpected behaviour. I've got three tokens in my sample terraform, and running terraform apply with no changes, I see this diff for one of them. This one has an expiry date, which doesnt change! but the diff thinks it does:

Terraform will perform the following actions:

  # grafana_cloud_private_data_source_connect_network_token.test3 must be replaced
-/+ resource "grafana_cloud_private_data_source_connect_network_token" "test3" {
      ~ created_at     = "2025-01-16T10:21:45Z" -> (known after apply)
      ~ expires_at     = "2025-02-01T00:00:00Z" -> "2025-02-01T00:00:00.00Z" # forces replacement
      ~ id             = "prod-gb-south-0:0a466813-3e6d-4e0d-ac09-2c6ba70b7669" -> (known after apply)
        name           = "my-pdc-token-3"
      ~ token          = (sensitive value)
      + updated_at     = (known after apply)
        # (3 unchanged attributes hidden)
    }

Plan: 1 to add, 0 to change, 1 to destroy.

I think once we've fixed this one we're good to go.

Copy link
Member

@spinillos spinillos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code looks like good to me

Copy link

@dafydd-t dafydd-t left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@aangelisc aangelisc merged commit 6918038 into main Jan 20, 2025
26 checks passed
@aangelisc aangelisc deleted the andreas/pdc-provider branch January 20, 2025 12:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support Private Datasource Connect (PDC) in Data Source Support Grafana Private Datasource Connect (PDC)
3 participants