Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Resource V2 SCC Findings Export to Big Query Folder Config #11517

Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
146 changes: 146 additions & 0 deletions mmv1/products/securitycenterv2/FolderSccBigQueryExports.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,146 @@
# Copyright 2024 Google Inc.
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

--- !ruby/object:Api::Resource
name: 'FolderSccBigQueryExports'
base_url: folders/{{folder}}/locations/{{location}}/bigQueryExports
self_link: folders/{{folder}}/locations/{{location}}/bigQueryExports/{{big_query_export_id}}
create_url: folders/{{folder}}/locations/{{location}}/bigQueryExports?bigQueryExportId={{big_query_export_id}}
update_verb: :PATCH
update_mask: true
import_format:
- 'folders/{{folder}}/locations/{{location}}/bigQueryExports/{{big_query_export_id}}'
custom_code: !ruby/object:Provider::Terraform::CustomCode
post_import: templates/terraform/post_import/scc_v2_folder_big_query_export_config.go.erb
description: |
A Cloud Security Command Center (Cloud SCC) Big Query Export Config.
It represents exporting Security Command Center data, including assets, findings, and security marks
using gcloud scc bqexports
~> **Note:** In order to use Cloud SCC resources, your organization must be enrolled
in [SCC Standard/Premium](https://cloud.google.com/security-command-center/docs/quickstart-security-command-center).
Without doing so, you may run into errors during resource creation.
references: !ruby/object:Api::Resource::ReferenceLinks
guides:
'Official Documentation': 'https://cloud.google.com/security-command-center/docs/how-to-analyze-findings-in-big-query'
api: 'https://cloud.google.com/security-command-center/docs/reference/rest/v2/folders.locations.bigQueryExports'
examples:
- !ruby/object:Provider::Terraform::Examples
name: 'scc_v2_folder_big_query_export_config_basic'
primary_resource_id: 'custom_big_query_export_config'
external_providers: ["random", "time"]
skip_test: true
vars:
big_query_export_id: 'my-export'
dataset: 'my-dataset'
dataset_id: 'my_dataset_id'
name: 'my-export'
folder_display_name: "folder-name"
test_env_vars:
org_id: :ORG_ID
project: :PROJECT_NAME

parameters:
- !ruby/object:Api::Type::String
name: folder
required: true
immutable: true
url_param_only: true
description: |
The folder where Cloud Security Command Center Big Query Export
Config lives in.
- !ruby/object:Api::Type::String
name: bigQueryExportId
required: true
immutable: true
url_param_only: true
description: |
This must be unique within the organization.
SarahFrench marked this conversation as resolved.
Show resolved Hide resolved
- !ruby/object:Api::Type::String
name: location
immutable: true
url_param_only: true
default_value: global
description: |
location Id is provided by organization. If not provided, Use global as default.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does the location value need to match the location of the destination dataset? I find the first part of this description a bit ambiguous

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

properties:
- !ruby/object:Api::Type::String
name: name
SarahFrench marked this conversation as resolved.
Show resolved Hide resolved
description: |
The resource name of this export, in the format
`folders/{{folder}}/locations/{{location}}/bigQueryExports/{{big_query_export_id}}`.
This field is provided in responses, and is ignored when provided in create requests.
- !ruby/object:Api::Type::String
name: description
description: |
The description of the notification config (max of 1024 characters).
validation: !ruby/object:Provider::Terraform::Validation
function: 'validation.StringLenBetween(0, 1024)'
- !ruby/object:Api::Type::String
name: dataset
description: |
The dataset to write findings' updates to.
Its format is "projects/[projectId]/datasets/[bigquery_dataset_id]".
BigQuery Dataset unique ID must contain only letters (a-z, A-Z), numbers (0-9), or underscores (_).
- !ruby/object:Api::Type::String
name: createTime
output: true
description: |
The time at which the BigQuery export was created. This field is set by the server and will be ignored if provided on export on creation.
A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits.
Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".
- !ruby/object:Api::Type::String
name: updateTime
output: true
description: |
The most recent time at which the BigQuery export was updated. This field is set by the server and will be ignored if provided on export creation or update.
A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits.
Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".
- !ruby/object:Api::Type::String
name: mostRecentEditor
output: true
description: |
Email address of the user who last edited the BigQuery export.
This field is set by the server and will be ignored if provided on export creation or update.
- !ruby/object:Api::Type::String
name: principal
output: true
description: |
The service account that needs permission to create table and upload data to the BigQuery dataset.
- !ruby/object:Api::Type::String
name: filter
description: |
Expression that defines the filter to apply across create/update
events of findings. The
expression is a list of zero or more restrictions combined via
logical operators AND and OR. Parentheses are supported, and OR
has higher precedence than AND.

Restrictions have the form <field> <operator> <value> and may have
a - character in front of them to indicate negation. The fields
map to those defined in the corresponding resource.

The supported operators are:

* = for all value types.
* >, <, >=, <= for integer values.
* :, meaning substring matching, for strings.

The supported value types are:

* string literals in quotes.
* integer literals without quotes.
* boolean literals true and false without quotes.

See
[Filtering notifications](https://cloud.google.com/security-command-center/docs/how-to-api-filter-notifications)
for information on how to write a filter.
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
resource "google_folder" "folder" {
parent = "organizations/<%= ctx[:test_env_vars]['org_id'] %>"
display_name = "<%= ctx[:vars]['folder_display_name'] %>"

deletion_protection = false
}

resource "google_bigquery_dataset" "default" {
dataset_id = "<%= ctx[:vars]['dataset_id'] %>"
friendly_name = "test"
description = "This is a test description"
location = "US"
default_table_expiration_ms = 3600000
default_partition_expiration_ms = null

labels = {
env = "default"
}

lifecycle {
ignore_changes = [default_partition_expiration_ms]
}
}

resource "google_scc_v2_folder_scc_big_query_exports" "<%= ctx[:primary_resource_id] %>" {
big_query_export_id = "<%= ctx[:vars]['big_query_export_id'] %>"
folder = google_folder.folder.folder_id
dataset = "<%= ctx[:vars]['dataset'] %>"
SarahFrench marked this conversation as resolved.
Show resolved Hide resolved
location = "global"
description = "Cloud Security Command Center Findings Big Query Export Config"
filter = "state=\"ACTIVE\" AND NOT mute=\"MUTED\""
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
idParts := strings.Split(d.Id(), "/")
SarahFrench marked this conversation as resolved.
Show resolved Hide resolved
if len(idParts) != 6 {
return nil, fmt.Errorf("unexpected format of ID (%q), expected folders/{{folder}}/locations/{{location}}/bigQueryExports/{{big_query_export_id}}", d.Id())
}

if err := d.Set("folder", idParts[1]); err != nil {
return nil, fmt.Errorf("error setting folder: %s", err)
}

if err := d.Set("big_query_export_id", idParts[5]); err != nil {
return nil, fmt.Errorf("error setting big_query_export_id: %s", err)
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,160 @@
package securitycenterv2_test

import (
"fmt"
SarahFrench marked this conversation as resolved.
Show resolved Hide resolved
"testing"

"github.com/hashicorp/terraform-plugin-testing/helper/resource"
"github.com/hashicorp/terraform-provider-google/google/acctest"
"github.com/hashicorp/terraform-provider-google/google/envvar"
)

func TestAccSecurityCenterV2FolderBigQueryExportConfig_basic(t *testing.T) {
t.Parallel()

randomSuffix := acctest.RandString(t, 10)
dataset_id := "tf_test_" + randomSuffix
orgID := envvar.GetTestOrgFromEnv(t)

context := map[string]interface{}{
"org_id": orgID,
"random_suffix": randomSuffix,
"dataset_id": dataset_id,
"dataset": fmt.Sprintf("projects/%s/datasets/%s",
envvar.GetTestProjectFromEnv(), dataset_id),
"big_query_export_id": "tf-test-export-" + randomSuffix,
"folder_name": "tf-test-folder-name-" + randomSuffix,
}

acctest.VcrTest(t, resource.TestCase{
PreCheck: func() { acctest.AccTestPreCheck(t) },
ProtoV5ProviderFactories: acctest.ProtoV5ProviderFactories(t),
ExternalProviders: map[string]resource.ExternalProvider{
"random": {},
"time": {},
},
Steps: []resource.TestStep{
{
Config: testAccSecurityCenterV2FolderBigQueryExportConfig_basic(context),
},
{
ResourceName: "google_scc_v2_folder_scc_big_query_exports.default",
ImportState: true,
ImportStateVerify: true,
ImportStateVerifyIgnore: []string{"update_time"},
},
{
Config: testAccSecurityCenterV2FolderBigQueryExportConfig_update(context),
},
{
ResourceName: "google_scc_v2_folder_scc_big_query_exports.default",
ImportState: true,
ImportStateVerify: true,
ImportStateVerifyIgnore: []string{"update_time"},
},
},
})
}

func testAccSecurityCenterV2FolderBigQueryExportConfig_basic(context map[string]interface{}) string {
return acctest.Nprintf(`

resource "google_folder" "folder" {
parent = "organizations/%{org_id}"
display_name = "%{folder_name}"

deletion_protection = false
}

resource "google_bigquery_dataset" "default" {
dataset_id = "%{dataset_id}"
friendly_name = "test"
description = "This is a test description"
location = "US"
default_table_expiration_ms = 3600000
default_partition_expiration_ms = null

labels = {
env = "default"
}

lifecycle {
ignore_changes = [default_partition_expiration_ms]
}
}

resource "time_sleep" "wait_1_minute" {
depends_on = [google_bigquery_dataset.default]
create_duration = "3m"
}

resource "google_scc_v2_folder_scc_big_query_exports" "default" {
big_query_export_id = "%{big_query_export_id}"
folder = google_folder.folder.folder_id
dataset = "%{dataset}"
location = "global"
description = "Cloud Security Command Center Findings Big Query Export Config"
filter = "state=\"ACTIVE\" AND NOT mute=\"MUTED\""

lifecycle {
ignore_changes = [name]
}

depends_on = [time_sleep.wait_1_minute]

}

resource "time_sleep" "wait_for_cleanup" {
create_duration = "3m"
depends_on = [google_scc_v2_folder_scc_big_query_exports.default]
}
`, context)
}

func testAccSecurityCenterV2FolderBigQueryExportConfig_update(context map[string]interface{}) string {
return acctest.Nprintf(`

resource "google_folder" "folder" {
parent = "organizations/%{org_id}"
display_name = "%{folder_name}"

deletion_protection = false
}

resource "google_bigquery_dataset" "default" {
dataset_id = "%{dataset_id}"
friendly_name = "test"
description = "This is a test description"
location = "US"
default_table_expiration_ms = 3600000
default_partition_expiration_ms = null

labels = {
env = "default"
}

lifecycle {
ignore_changes = [default_partition_expiration_ms]
}
}

resource "google_scc_v2_folder_scc_big_query_exports" "default" {
big_query_export_id = "%{big_query_export_id}"
folder = google_folder.folder.folder_id
dataset = "%{dataset}"
location = "global"
description = "SCC Findings Big Query Export Update"
filter = "state=\"ACTIVE\" AND NOT mute=\"MUTED\""

lifecycle {
ignore_changes = [name]
}

}

resource "time_sleep" "wait_for_cleanup" {
create_duration = "3m"
depends_on = [google_scc_v2_folder_scc_big_query_exports.default]
}
`, context)
}