Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable $web and disable anonymous blob access on rpversions container #4074

Merged
merged 7 commits into from
Feb 5, 2025
Merged

Enable $web and disable anonymous blob access on rpversions container #4074

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
276248d
Enable static web content on versions SA
tsatam Jan 31, 2025
f75f3c3
Store deploy version information in web container
tsatam Jan 31, 2025
fb6db60
Disable anonymous access on version SA and delete dedicated rp/ocp ve…
tsatam Jan 31, 2025
4d555a5
Update generated rp-production-global ARM template
tsatam Jan 31, 2025
fe64a96
Set StorageAccount Kind to StorageV2; Min TLS to 1.2 in ARM template
tsatam Jan 31, 2025
5d4dad0
Switch to using track2 blob sdk for saveversion
tsatam Feb 4, 2025
784965e
Remove unused/deprecated storage sdk
tsatam Feb 4, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 0 additions & 2 deletions go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -194,8 +194,6 @@ github.com/disiqueira/gotree/v3 v3.0.2 h1:ik5iuLQQoufZBNPY518dXhiO5056hyNBIK9lWh
github.com/disiqueira/gotree/v3 v3.0.2/go.mod h1:ZuyjE4+mUQZlbpkI24AmruZKhg3VHEgPLDY8Qk+uUu8=
github.com/distribution/reference v0.6.0 h1:0IXCQ5g4/QMHHkarYzh5l+u8T3t73zM5QvfrDyIgxBk=
github.com/distribution/reference v0.6.0/go.mod h1:BbU0aIcezP1/5jX/8MP0YiH4SdvB5Y4f/wlDRiLyi3E=
github.com/dnaeon/go-vcr v1.2.0 h1:zHCHvJYTMh1N7xnV7zf1m1GPBF9Ad0Jk/whtQ1663qI=
github.com/dnaeon/go-vcr v1.2.0/go.mod h1:R4UdLID7HZT3taECzJs4YgbbH6PIGXB6W/sc5OLb6RQ=
github.com/docker/cli v26.1.3+incompatible h1:bUpXT/N0kDE3VUHI2r5VMsYQgi38kYuoC0oL9yt3lqc=
github.com/docker/cli v26.1.3+incompatible/go.mod h1:JLrzqnKDaYBop7H2jaqPtU4hHvMKP+vjCwu2uszcLI8=
github.com/docker/distribution v2.8.2+incompatible h1:T3de5rq0dB1j30rp0sA2rER+m322EBzniBPB6ZIzuh8=
Expand Down
32 changes: 4 additions & 28 deletions pkg/deploy/assets/rp-production-global.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -101,40 +101,16 @@
"sku": {
"name": "Standard_LRS"
},
"kind": "StorageV2",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: You mentioned in the walk though that if there was ever a reason to use a different version or expand our usage of this we may need to change this. Is there a code comment we should leave for future generations?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the required code change should be fairly self-explanatory, and would also be dependent on whatever the reason for changing/extending this would be. I also don't think there's actually any compelling reason for us to need any kind other than StorageV2 in the future, but unforeseen circumstances might require that.

Regarding TLS, this property is just setting the "minimum" supported TLS version, the highest we can even set this to is 1.2 currently anyways (https://learn.microsoft.com/en-us/azure/storage/common/transport-layer-security-configure-minimum-version?tabs=azure-cli#configure-the-minimum-tls-version-for-a-storage-account).

"properties": {
"allowBlobPublicAccess": true
},
"tags": {
"Az.Sec.AnonymousBlobAccessEnforcement::Skip": "PublicRelease"
"allowBlobPublicAccess": false,
"minimumTlsVersion": "TLS1_2"
},
"tags": {},
"location": "[resourceGroup().location]",
"name": "[parameters('rpVersionStorageAccountName')]",
"type": "Microsoft.Storage/storageAccounts",
"apiVersion": "2021-09-01"
},
{
"properties": {
"publicAccess": "Container",
"metadata": null
},
"name": "[concat(parameters('rpVersionStorageAccountName'), '/default/rpversion')]",
"type": "Microsoft.Storage/storageAccounts/blobServices/containers",
"apiVersion": "2021-09-01",
"dependsOn": [
"[resourceId('Microsoft.Storage/storageAccounts', parameters('rpVersionStorageAccountName'))]"
]
},
{
"properties": {
"publicAccess": "Container",
"metadata": null
},
"name": "[concat(parameters('rpVersionStorageAccountName'), '/default/ocpversions')]",
"type": "Microsoft.Storage/storageAccounts/blobServices/containers",
"apiVersion": "2021-09-01",
"dependsOn": [
"[resourceId('Microsoft.Storage/storageAccounts', parameters('rpVersionStorageAccountName'))]"
]
}
]
}
1 change: 1 addition & 0 deletions pkg/deploy/generator/resources.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,7 @@ func (g *generator) storageAccount(name string, accountProperties *mgmtstorage.A
Resource: &mgmtstorage.Account{
Name: &name,
Type: to.StringPtr("Microsoft.Storage/storageAccounts"),
Kind: mgmtstorage.KindStorageV2,
Location: to.StringPtr("[resourceGroup().location]"),
Sku: &mgmtstorage.Sku{
Name: "Standard_LRS",
Expand Down
37 changes: 7 additions & 30 deletions pkg/deploy/generator/resources_rp.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1518,36 +1518,13 @@ func (g *generator) rpACRRBAC() []*arm.Resource {

func (g *generator) rpVersionStorageAccount() []*arm.Resource {
return []*arm.Resource{
g.storageAccount("[parameters('rpVersionStorageAccountName')]", &mgmtstorage.AccountProperties{
AllowBlobPublicAccess: to.BoolPtr(true),
}, map[string]*string{
tagKeyExemptPublicBlob: to.StringPtr(tagValueExemptPublicBlob),
}),
{
Resource: &mgmtstorage.BlobContainer{
Name: to.StringPtr("[concat(parameters('rpVersionStorageAccountName'), '/default/rpversion')]"),
Type: to.StringPtr("Microsoft.Storage/storageAccounts/blobServices/containers"),
ContainerProperties: &mgmtstorage.ContainerProperties{
PublicAccess: mgmtstorage.PublicAccessContainer,
},
},
APIVersion: azureclient.APIVersion("Microsoft.Storage"),
DependsOn: []string{
"[resourceId('Microsoft.Storage/storageAccounts', parameters('rpVersionStorageAccountName'))]",
},
},
{
Resource: &mgmtstorage.BlobContainer{
Name: to.StringPtr("[concat(parameters('rpVersionStorageAccountName'), '/default/ocpversions')]"),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm fairly confident we're not using OCPVersions at all, but can you confirm? I'm pretty certain they're stale.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For context/history (if i'm remembering correctly), we used this back when we had a single version install target and didn't support multiple versions.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I checked our prod container and the newest OCP version it had was a 4.11.z version, so I hope we still aren't using it for anything

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's also worth noting that the absence of these blobcontainer resources doesn't result in existing containers being deleted, they're just left alone (verified in int). If we do want to delete these resources we'll likely need to add an explicit fixup step during deployment.

Type: to.StringPtr("Microsoft.Storage/storageAccounts/blobServices/containers"),
ContainerProperties: &mgmtstorage.ContainerProperties{
PublicAccess: mgmtstorage.PublicAccessContainer,
},
},
APIVersion: azureclient.APIVersion("Microsoft.Storage"),
DependsOn: []string{
"[resourceId('Microsoft.Storage/storageAccounts', parameters('rpVersionStorageAccountName'))]",
g.storageAccount(
"[parameters('rpVersionStorageAccountName')]",
&mgmtstorage.AccountProperties{
AllowBlobPublicAccess: to.BoolPtr(false),
MinimumTLSVersion: mgmtstorage.MinimumTLSVersionTLS12,
},
},
map[string]*string{},
),
}
}
42 changes: 29 additions & 13 deletions pkg/deploy/saveversion.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,25 +4,27 @@ package deploy
// Licensed under the Apache License 2.0.

import (
"bytes"
"context"
"net/url"
"fmt"
"time"

"github.com/Azure/azure-sdk-for-go/sdk/storage/azblob/service"
mgmtstorage "github.com/Azure/azure-sdk-for-go/services/storage/mgmt/2021-09-01/storage"
azstorage "github.com/Azure/azure-sdk-for-go/storage"
"github.com/Azure/go-autorest/autorest/date"

"github.com/Azure/ARO-RP/pkg/util/azureclient/azuresdk/azblob"
"github.com/Azure/ARO-RP/pkg/util/pointerutils"
tsatam marked this conversation as resolved.
Show resolved Hide resolved
)

// SaveVersion for current location in shared storage account for environment
func (d *deployer) SaveVersion(ctx context.Context) error {
d.log.Printf("saving RP and OCP versions for RP %s deployed in %s to storage account %s", d.version, d.config.Location, *d.config.Configuration.RPVersionStorageAccountName)
d.log.Printf("saving RP version %s deployed in %s to storage account %s", d.version, d.config.Location, *d.config.Configuration.RPVersionStorageAccountName)
t := time.Now().UTC().Truncate(time.Second)
res, err := d.globalaccounts.ListAccountSAS(
ctx, *d.config.Configuration.GlobalResourceGroupName, *d.config.Configuration.RPVersionStorageAccountName, mgmtstorage.AccountSasParameters{
Services: mgmtstorage.ServicesB,
ResourceTypes: mgmtstorage.SignedResourceTypesO,
Permissions: "cw", // create and write
ResourceTypes: mgmtstorage.SignedResourceTypesO + mgmtstorage.SignedResourceTypesS,
Permissions: mgmtstorage.PermissionsC + mgmtstorage.PermissionsW, // create and write
Protocols: mgmtstorage.HTTPProtocolHTTPS,
SharedAccessStartTime: &date.Time{Time: t},
SharedAccessExpiryTime: &date.Time{Time: t.Add(24 * time.Hour)},
Expand All @@ -31,16 +33,30 @@ func (d *deployer) SaveVersion(ctx context.Context) error {
return err
}

v, err := url.ParseQuery(*res.AccountSasToken)
d.log.Infof("instantiating blobs client using SAS token")
sasUrl := fmt.Sprintf("https://%s.blob.%s/?%s", *d.config.Configuration.RPVersionStorageAccountName, d.env.Environment().StorageEndpointSuffix, *res.AccountSasToken)
blobsClient, err := azblob.NewBlobsClientUsingSAS(sasUrl, d.env.Environment().ArmClientOptions())
if err != nil {
d.log.Errorf("failure to instantiate blobs client using SAS: %v", err)
return err
}

d.log.Infof("ensuring static web content is enabled")
_, err = blobsClient.ServiceClient().SetProperties(ctx, &service.SetPropertiesOptions{
StaticWebsite: &service.StaticWebsite{Enabled: pointerutils.ToPtr(true)},
})
if err != nil {
d.log.Errorf("failure to update static properties: %v", err)
return err
}

blobClient := azstorage.NewAccountSASClient(
*d.config.Configuration.RPVersionStorageAccountName, v, (*d.env.Environment()).Environment).GetBlobService()
d.log.Infof("uploading RP version")
blobName := fmt.Sprintf("rpversion/%s", d.config.Location)
_, err = blobsClient.UploadBuffer(ctx, "$web", blobName, []byte(d.version), nil)
if err != nil {
d.log.Errorf("failure to upload version information: %v", err)
return err
}

// save version of RP which is deployed in this location
containerRef := blobClient.GetContainerReference("rpversion")
blobRef := containerRef.GetBlobReference(d.config.Location)
return blobRef.CreateBlockBlobFromReader(bytes.NewReader([]byte(d.version)), nil)
return nil
}
23 changes: 0 additions & 23 deletions vendor/github.com/Azure/azure-sdk-for-go/storage/README.md
View file
Open in desktop

This file was deleted.

79 changes: 0 additions & 79 deletions vendor/github.com/Azure/azure-sdk-for-go/storage/appendblob.go
View file
Open in desktop

This file was deleted.

Loading
Loading