Update LoadSecrets() #7796

lakshmimsft · 2024-08-09T18:55:07Z

Description

Update LoadSecrets() to return data for all keys for corresponding secret store when no keys filter is provided.
Update return type to include SecretData{}
Updated existing LoadSecrets() to return data collated for multiple secret store ids
Update to helper function populateSecretData() and it's unit tests which populates secret data map returned.
Update functional test to include envSecrets input

Type of change

This pull request adds or changes features of Radius and has an approved issue (Recipe support for private Bicep registries #6917).

Fixes: (Part of #6917)

lakshmimsft · 2024-08-09T18:56:53Z

pkg/recipes/configloader/secrets.go

+		if secretData[secretStoreID] == nil {
+			secretData[secretStoreID] = make(map[string]string)
+		}
+		secretData[secretStoreID][secretKey] = *secretDataValue.Value


ln96- ln105 is just ln76-ln82 refactored

codecov · 2024-08-09T19:06:53Z

Codecov Report

Attention: Patch coverage is 84.84848% with 5 lines in your changes missing coverage. Please review.

Project coverage is 61.06%. Comparing base (a58b8cb) to head (c151322).
Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
pkg/recipes/configloader/secrets.go	76.19%	5 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #7796      +/-   ##
==========================================
+ Coverage   61.05%   61.06%   +0.01%     
==========================================
  Files         523      523              
  Lines       27466    27474       +8     
==========================================
+ Hits        16770    16778       +8     
- Misses       9211     9212       +1     
+ Partials     1485     1484       -1

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

radius-functional-tests · 2024-08-09T19:24:12Z

Radius functional test overview

🔍 Go to test action run

Name	Value
Repository	lakshmimsft/radius-mainfork
Commit ref	`e8e4385`
Unique ID	func017672a42d
Image tag	pr-func017672a42d

Click here to see the list of tools in the current test run

gotestsum 1.12.0
KinD: v0.20.0
Dapr: 1.12.0
Azure KeyVault CSI driver: 1.4.2
Azure Workload identity webhook: 1.1.0
Bicep recipe location ghcr.io/radius-project/dev/test/testrecipes/test-bicep-recipes/<name>:pr-func017672a42d
Terraform recipe location http://tf-module-server.radius-test-tf-module-server.svc.cluster.local/<name>.zip (in cluster)
applications-rp test image location: ghcr.io/radius-project/dev/applications-rp:pr-func017672a42d
controller test image location: ghcr.io/radius-project/dev/controller:pr-func017672a42d
ucp test image location: ghcr.io/radius-project/dev/ucpd:pr-func017672a42d
deployment-engine test image location: ghcr.io/radius-project/deployment-engine:latest

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting corerp-cloud functional tests...
⌛ Starting datastoresrp-cloud functional tests...
⌛ Starting ucp-cloud functional tests...
✅ ucp-cloud functional tests succeeded
✅ datastoresrp-cloud functional tests succeeded
✅ corerp-cloud functional tests succeeded

radius-functional-tests · 2024-08-13T23:24:44Z

Radius functional test overview

🔍 Go to test action run

Name	Value
Repository	lakshmimsft/radius-mainfork
Commit ref	`5ed6808`
Unique ID	func09d6d3a6f7
Image tag	pr-func09d6d3a6f7

Click here to see the list of tools in the current test run

gotestsum 1.12.0
KinD: v0.20.0
Dapr: 1.12.0
Azure KeyVault CSI driver: 1.4.2
Azure Workload identity webhook: 1.3.0
Bicep recipe location ghcr.io/radius-project/dev/test/testrecipes/test-bicep-recipes/<name>:pr-func09d6d3a6f7
Terraform recipe location http://tf-module-server.radius-test-tf-module-server.svc.cluster.local/<name>.zip (in cluster)
applications-rp test image location: ghcr.io/radius-project/dev/applications-rp:pr-func09d6d3a6f7
controller test image location: ghcr.io/radius-project/dev/controller:pr-func09d6d3a6f7
ucp test image location: ghcr.io/radius-project/dev/ucpd:pr-func09d6d3a6f7
deployment-engine test image location: ghcr.io/radius-project/deployment-engine:latest

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting corerp-cloud functional tests...
⌛ Starting datastoresrp-cloud functional tests...
⌛ Starting ucp-cloud functional tests...
✅ datastoresrp-cloud functional tests succeeded
✅ corerp-cloud functional tests succeeded
✅ ucp-cloud functional tests succeeded

ytimocin · 2024-08-15T21:18:57Z

pkg/recipes/configloader/secrets.go

+// If the input secret keys filter for a secret store ID is nil or empty, it retrieves secret data for all keys for that secret store.
+// When the secret keys filter is populated for a secret store ID, it retrieves secret data for the specified keys for the associated secret store ID.
+// The function returns a map of secret data, where the keys are the secret store IDs and the values are maps of secret keys and their corresponding values.
+func (e *secretsLoader) LoadSecrets(ctx context.Context, secretStoreIDKeysFilter map[string][]string) (secretData map[string]map[string]string, err error) {


Why not just name this filter and give an example in the doc above?

Suggested change

func (e *secretsLoader) LoadSecrets(ctx context.Context, secretStoreIDKeysFilter map[string][]string) (secretData map[string]map[string]string, err error) {

func (e *secretsLoader) LoadSecrets(ctx context.Context, filter map[string][]string) (secretData map[string]map[string]string, err error) {

pkg/recipes/configloader/secrets.go

pkg/recipes/configloader/secrets_test.go

pkg/recipes/configloader/secrets.go

radius-functional-tests · 2024-08-22T16:35:47Z

Radius functional test overview

🔍 Go to test action run

Name	Value
Repository	lakshmimsft/radius-mainfork
Commit ref	`cbf1cbc`
Unique ID	funcf15ed1e337
Image tag	pr-funcf15ed1e337

Click here to see the list of tools in the current test run

gotestsum 1.12.0
KinD: v0.20.0
Dapr: 1.12.0
Azure KeyVault CSI driver: 1.4.2
Azure Workload identity webhook: 1.3.0
Bicep recipe location ghcr.io/radius-project/dev/test/testrecipes/test-bicep-recipes/<name>:pr-funcf15ed1e337
Terraform recipe location http://tf-module-server.radius-test-tf-module-server.svc.cluster.local/<name>.zip (in cluster)
applications-rp test image location: ghcr.io/radius-project/dev/applications-rp:pr-funcf15ed1e337
controller test image location: ghcr.io/radius-project/dev/controller:pr-funcf15ed1e337
ucp test image location: ghcr.io/radius-project/dev/ucpd:pr-funcf15ed1e337
deployment-engine test image location: ghcr.io/radius-project/deployment-engine:latest

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting datastoresrp-cloud functional tests...
✅ datastoresrp-cloud functional tests succeeded
⌛ Starting corerp-cloud functional tests...
⌛ Starting ucp-cloud functional tests...
✅ corerp-cloud functional tests succeeded
✅ ucp-cloud functional tests succeeded

pkg/recipes/terraform/types.go

vishwahiremat · 2024-08-22T17:03:10Z

...tional-portable/corerp/noncloud/resources/testdata/corerp-resources-terraform-postgres.bicep

@@ -24,13 +24,15 @@ resource env 'Applications.Core/environments@2023-10-01-preview' = {
        providers: {


are these changes related to this PR?

Discussed offline. The functional test needs to be updated with envSecrets block. This PR updates code that can affect running of functional test with envSecrets. We'll include this test in the PR.

vishwahiremat · 2024-08-22T17:05:45Z

pkg/recipes/terraform/execute.go

@@ -225,7 +225,7 @@ func (e executor) setEnvironmentVariables(tf *tfexec.Terraform, options Options)
 		for secretName, secretReference := range recipeConfig.EnvSecrets {
 			// Extract secret value from the secrets input
 			if secretIDs, ok := options.Secrets[secretReference.Source]; ok {
-				if secretValue, ok := secretIDs[secretReference.Key]; ok {
+				if secretValue, ok := secretIDs.Data[secretReference.Key]; ok {
 					envVarUpdate = true


id it gonna throw nil pointer if secretIDs is nil?

secretIDs is a struct and will not be nil. Renamed variable to secretData. There exists a unit test to check for secretData.Data when nil.

vishwahiremat · 2024-08-26T06:34:49Z

bicep-types

are these changes related to this PR?

removed from PR.

pkg/recipes/configloader/secrets.go

radius-functional-tests · 2024-08-27T23:56:42Z

Radius functional test overview

🔍 Go to test action run

Name	Value
Repository	lakshmimsft/radius-mainfork
Commit ref	`93f111b`
Unique ID	func4f779ac8c1
Image tag	pr-func4f779ac8c1

Click here to see the list of tools in the current test run

gotestsum 1.12.0
KinD: v0.20.0
Dapr: 1.12.0
Azure KeyVault CSI driver: 1.4.2
Azure Workload identity webhook: 1.3.0
Bicep recipe location ghcr.io/radius-project/dev/test/testrecipes/test-bicep-recipes/<name>:pr-func4f779ac8c1
Terraform recipe location http://tf-module-server.radius-test-tf-module-server.svc.cluster.local/<name>.zip (in cluster)
applications-rp test image location: ghcr.io/radius-project/dev/applications-rp:pr-func4f779ac8c1
controller test image location: ghcr.io/radius-project/dev/controller:pr-func4f779ac8c1
ucp test image location: ghcr.io/radius-project/dev/ucpd:pr-func4f779ac8c1
deployment-engine test image location: ghcr.io/radius-project/deployment-engine:latest

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting datastoresrp-cloud functional tests...
⌛ Starting ucp-cloud functional tests...
⌛ Starting corerp-cloud functional tests...
✅ datastoresrp-cloud functional tests succeeded
✅ ucp-cloud functional tests succeeded
❌ corerp-cloud functional test failed. Please check the logs for more details
⌛ Starting corerp-cloud functional tests...
❌ corerp-cloud functional test failed. Please check the logs for more details
⌛ Starting corerp-cloud functional tests...
❌ corerp-cloud functional test failed. Please check the logs for more details

vishwahiremat

lgtm

radius-functional-tests · 2024-08-29T17:27:00Z

Radius functional test overview

🔍 Go to test action run

Name	Value
Repository	lakshmimsft/radius-mainfork
Commit ref	`c151322`
Unique ID	funcf1269ca09b
Image tag	pr-funcf1269ca09b

Click here to see the list of tools in the current test run

gotestsum 1.12.0
KinD: v0.20.0
Dapr: 1.12.0
Azure KeyVault CSI driver: 1.4.2
Azure Workload identity webhook: 1.3.0
Bicep recipe location ghcr.io/radius-project/dev/test/testrecipes/test-bicep-recipes/<name>:pr-funcf1269ca09b
Terraform recipe location http://tf-module-server.radius-test-tf-module-server.svc.cluster.local/<name>.zip (in cluster)
applications-rp test image location: ghcr.io/radius-project/dev/applications-rp:pr-funcf1269ca09b
controller test image location: ghcr.io/radius-project/dev/controller:pr-funcf1269ca09b
ucp test image location: ghcr.io/radius-project/dev/ucpd:pr-funcf1269ca09b
deployment-engine test image location: ghcr.io/radius-project/deployment-engine:latest

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting datastoresrp-cloud functional tests...
⌛ Starting corerp-cloud functional tests...
⌛ Starting ucp-cloud functional tests...
✅ datastoresrp-cloud functional tests succeeded
✅ ucp-cloud functional tests succeeded
✅ corerp-cloud functional tests succeeded

lakshmimsft commented Aug 9, 2024

View reviewed changes

lakshmimsft temporarily deployed to functional-tests August 9, 2024 19:23 — with GitHub Actions Inactive

lakshmimsft force-pushed the lakshmimsft/updtoLoadSecrets branch from e8e4385 to 5ed6808 Compare August 13, 2024 23:22

lakshmimsft marked this pull request as ready for review August 13, 2024 23:22

lakshmimsft requested review from a team as code owners August 13, 2024 23:22

lakshmimsft temporarily deployed to functional-tests August 13, 2024 23:23 — with GitHub Actions Inactive

lakshmimsft requested review from vishwahiremat and ytimocin August 13, 2024 23:28

lakshmimsft force-pushed the lakshmimsft/updtoLoadSecrets branch from 5ed6808 to 6f13718 Compare August 15, 2024 20:52

ytimocin reviewed Aug 15, 2024

View reviewed changes

vishwahiremat reviewed Aug 20, 2024

View reviewed changes

pkg/recipes/configloader/secrets.go Outdated Show resolved Hide resolved

vishwahiremat reviewed Aug 20, 2024

View reviewed changes

pkg/recipes/configloader/secrets.go Show resolved Hide resolved

lakshmimsft force-pushed the lakshmimsft/updtoLoadSecrets branch from fbbad7c to 1aec7a4 Compare August 22, 2024 16:15

lakshmimsft changed the title ~~Update LoadSecrets() to return data for all keys for corresponding secret store when no keys filter is provided~~ Update LoadSecrets() Aug 22, 2024

lakshmimsft temporarily deployed to functional-tests August 22, 2024 16:35 — with GitHub Actions Inactive

vishwahiremat reviewed Aug 26, 2024

View reviewed changes

lakshmimsft force-pushed the lakshmimsft/updtoLoadSecrets branch 3 times, most recently from 1aec7a4 to 93f111b Compare August 27, 2024 23:33

lakshmimsft temporarily deployed to functional-tests August 27, 2024 23:55 — with GitHub Actions Inactive

vishwahiremat approved these changes Aug 28, 2024

View reviewed changes

lakshmimsft added 3 commits August 29, 2024 10:13

update LoadSecrets func

3f41421

updating func tests

0e813aa

updated return type

60abb4f

update per comments

c151322

lakshmimsft force-pushed the lakshmimsft/updtoLoadSecrets branch from 93f111b to c151322 Compare August 29, 2024 17:13

lakshmimsft temporarily deployed to functional-tests August 29, 2024 17:26 — with GitHub Actions Inactive

ytimocin approved these changes Aug 29, 2024

View reviewed changes

lakshmimsft merged commit 84cb120 into radius-project:main Aug 29, 2024
28 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update LoadSecrets() #7796

Update LoadSecrets() #7796

lakshmimsft commented Aug 9, 2024 •

edited

Loading

lakshmimsft Aug 9, 2024

codecov bot commented Aug 9, 2024 •

edited

Loading

radius-functional-tests bot commented Aug 9, 2024 •

edited

Loading

radius-functional-tests bot commented Aug 13, 2024 •

edited

Loading

ytimocin Aug 15, 2024

radius-functional-tests bot commented Aug 22, 2024 •

edited

Loading

vishwahiremat Aug 22, 2024

lakshmimsft Aug 27, 2024

vishwahiremat Aug 22, 2024

lakshmimsft Aug 27, 2024

vishwahiremat Aug 26, 2024

lakshmimsft Aug 27, 2024

radius-functional-tests bot commented Aug 27, 2024 •

edited

Loading

vishwahiremat left a comment

radius-functional-tests bot commented Aug 29, 2024 •

edited

Loading

	func (e *secretsLoader) LoadSecrets(ctx context.Context, secretStoreIDKeysFilter map[string][]string) (secretData map[string]map[string]string, err error) {
	func (e *secretsLoader) LoadSecrets(ctx context.Context, filter map[string][]string) (secretData map[string]map[string]string, err error) {

		@@ -24,13 +24,15 @@ resource env 'Applications.Core/environments@2023-10-01-preview' = {
		providers: {

Update LoadSecrets() #7796

Update LoadSecrets() #7796

Conversation

lakshmimsft commented Aug 9, 2024 • edited Loading

Description

Type of change

Choose a reason for hiding this comment

codecov bot commented Aug 9, 2024 • edited Loading

Codecov Report

radius-functional-tests bot commented Aug 9, 2024 • edited Loading

Radius functional test overview

Test Status

radius-functional-tests bot commented Aug 13, 2024 • edited Loading

Radius functional test overview

Test Status

Choose a reason for hiding this comment

radius-functional-tests bot commented Aug 22, 2024 • edited Loading

Radius functional test overview

Test Status

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

radius-functional-tests bot commented Aug 27, 2024 • edited Loading

Radius functional test overview

Test Status

vishwahiremat left a comment

Choose a reason for hiding this comment

radius-functional-tests bot commented Aug 29, 2024 • edited Loading

Radius functional test overview

Test Status

lakshmimsft commented Aug 9, 2024 •

edited

Loading

codecov bot commented Aug 9, 2024 •

edited

Loading

radius-functional-tests bot commented Aug 9, 2024 •

edited

Loading

radius-functional-tests bot commented Aug 13, 2024 •

edited

Loading

radius-functional-tests bot commented Aug 22, 2024 •

edited

Loading

radius-functional-tests bot commented Aug 27, 2024 •

edited

Loading

radius-functional-tests bot commented Aug 29, 2024 •

edited

Loading