Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for client assertion based authentication #1620

Merged
merged 29 commits into from
Feb 24, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
3b14949
feat: support workload identity token (#1556)
cvvz Dec 5, 2024
dc504f6
Create block pool only once in child process (#1581)
souravgupta-msft Dec 10, 2024
c4e57ef
Update golang.org/x/crypto to v0.31.0 (#1594)
JanJaguschQC Dec 18, 2024
622e3d7
sync with main (#1603)
ashruti-msft Jan 3, 2025
dab05d9
Synch with main
vibhansa-msft Jan 6, 2025
75de959
Use ListBlob for hns accounts (#1555)
ashruti-msft Jan 15, 2025
c16d3d2
Added statfs for block-cache (#1470)
ashruti-msft Jan 15, 2025
37149a5
Add strong consistency check for data on disk (#1604)
vibhansa-msft Jan 15, 2025
cdd48de
bug in block cache open call (#1580)
syeleti-msft Jan 15, 2025
6c4cf2e
Feature: Blob filter (#1595)
vibhansa-msft Jan 16, 2025
76b69ad
Cleanup on start shall be set to cleanup temp cache (#1613)
vibhansa-msft Jan 20, 2025
c576f39
Automate blobfuse2 setup for new VM (#1575)
ashruti-msft Jan 24, 2025
bf1e726
Downgrade go version to 1.22.7 due to memory issues in 1.23 (#1619)
vibhansa-msft Jan 28, 2025
c71ea55
Enable ETAG based validation on every block download to provide highe…
vibhansa-msft Jan 30, 2025
0090d83
Adding Client assertion based authentication mode
vibhansa-msft Jan 30, 2025
99e2377
Adding client assertion and on behalf assertion auth mode
vibhansa-msft Jan 31, 2025
acc51fe
Remove onbehalf config
vibhansa-msft Jan 31, 2025
ddd419e
Correcting condition check
vibhansa-msft Jan 31, 2025
f97a7b9
Correcting condition check
vibhansa-msft Jan 31, 2025
0cc9045
Remove security scope option as resource string is already present
vibhansa-msft Jan 31, 2025
dc37f34
Correcting scope
vibhansa-msft Jan 31, 2025
49fdb82
Correcting scope
vibhansa-msft Jan 31, 2025
59bc9a6
sync with main branch
vibhansa-msft Feb 4, 2025
016a31f
Update version
vibhansa-msft Feb 4, 2025
ad04d9b
Merge branch 'main' into vibhansa/clientAssertion
vibhansa-msft Feb 10, 2025
7f1efbd
Merge remote-tracking branch 'origin/main' into vibhansa/clientAssertion
vibhansa-msft Feb 19, 2025
d79612b
Correcting default scope for token exchange
vibhansa-msft Feb 21, 2025
0741b58
Add code to ignore client assertion in code coverage
vibhansa-msft Feb 21, 2025
8be355f
Merge branch 'main' into vibhansa/clientAssertion
vibhansa-msft Feb 22, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
## 2.5.0 (Unreleased)
**Features**
- Added `Client Assertion` based authentication for containers. Configure `tenant-id, client-id, aad-application-id and security scope` with `authMode` set to `clientassertion`.

## 2.5.0~preview.1 (Unreleased)
**Features**
Expand Down
2 changes: 1 addition & 1 deletion blobfuse2-code-coverage.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -565,7 +565,7 @@ stages:
- script: |
echo 'mode: count' > ./blobfuse2_coverage_raw.rpt
tail -q -n +2 ./*.cov >> ./blobfuse2_coverage_raw.rpt
cat ./blobfuse2_coverage_raw.rpt | grep -v mock_component | grep -v base_component | grep -v loopback | grep -v tools | grep -v "common/log" | grep -v "common/exectime" | grep -v "common/types.go" | grep -v "internal/stats_manager" | grep -v "main.go" | grep -v "component/azstorage/azauthmsi.go" | grep -v "component/azstorage/azauthspn.go" | grep -v "component/stream" | grep -v "component/custom" | grep -v "component/azstorage/azauthcli.go" | grep -v "exported/exported.go" | grep -v "component/block_cache/stream.go" > ./blobfuse2_coverage.rpt
cat ./blobfuse2_coverage_raw.rpt | grep -v mock_component | grep -v base_component | grep -v loopback | grep -v tools | grep -v "common/log" | grep -v "common/exectime" | grep -v "common/types.go" | grep -v "internal/stats_manager" | grep -v "main.go" | grep -v "component/azstorage/azauthmsi.go" | grep -v "component/azstorage/azauthspn.go" | grep -v "component/stream" | grep -v "component/custom" | grep -v "component/azstorage/azauthcli.go" | grep -v "exported/exported.go" | grep -v "component/block_cache/stream.go" | grep -v "component/azstorage/azauthclientassertion.go" > ./blobfuse2_coverage.rpt
go tool cover -func blobfuse2_coverage.rpt > ./blobfuse2_func_cover.rpt
go tool cover -html=./blobfuse2_coverage.rpt -o ./blobfuse2_coverage.html
go tool cover -html=./blobfuse2_ut.cov -o ./blobfuse2_ut.html
Expand Down
2 changes: 1 addition & 1 deletion common/types.go
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ import (

// Standard config default values
const (
blobfuse2Version_ = "2.4.1"
blobfuse2Version_ = "2.5.0~preview.1"

DefaultMaxLogFileSize = 512
DefaultLogFileCount = 10
Expand Down
20 changes: 19 additions & 1 deletion component/azstorage/azauth.go
Original file line number Diff line number Diff line change
Expand Up @@ -66,8 +66,14 @@ type azAuthConfig struct {
WorkloadIdentityToken string
ActiveDirectoryEndpoint string

Endpoint string
// Client assertions config
// This will need ApplicationID, TenantID and ClientID as well
UserAssertion string

// Auth resource / security scope for OAuth
AuthResource string

Endpoint string
}

// azAuth : Interface to define a generic authentication type
Expand Down Expand Up @@ -131,6 +137,12 @@ func getAzBlobAuth(config azAuthConfig) azAuth {
azAuthBase: base,
},
}
} else if config.AuthMode == EAuthType.CLIENTASSERTION() {
return &azAuthBlobClientAssertion{
azAuthClientAssertion{
azAuthBase: base,
},
}
} else {
log.Crit("azAuth::getAzBlobAuth : Auth type %s not supported. Failed to create Auth object", config.AuthMode)
}
Expand Down Expand Up @@ -169,6 +181,12 @@ func getAzDatalakeAuth(config azAuthConfig) azAuth {
azAuthBase: base,
},
}
} else if config.AuthMode == EAuthType.CLIENTASSERTION() {
return &azAuthDatalakeClientAssertion{
azAuthClientAssertion{
azAuthBase: base,
},
}
} else {
log.Crit("azAuth::getAzDatalakeAuth : Auth type %s not supported. Failed to create Auth object", config.AuthMode)
}
Expand Down
162 changes: 162 additions & 0 deletions component/azstorage/azauthclientassertion.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,162 @@
/*
_____ _____ _____ ____ ______ _____ ------
| | | | | | | | | | | | |
| | | | | | | | | | | | |
| --- | | | | |-----| |---- | | |-----| |----- ------
| | | | | | | | | | | | |
| ____| |_____ | ____| | ____| | |_____| _____| |_____ |_____


Licensed under the MIT License <http://opensource.org/licenses/MIT>.

Copyright Β© 2020-2025 Microsoft Corporation. All rights reserved.
Author : <[email protected]>

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE
*/

package azstorage

import (
"context"

"github.com/Azure/azure-sdk-for-go/sdk/azcore"
"github.com/Azure/azure-sdk-for-go/sdk/azcore/policy"
"github.com/Azure/azure-sdk-for-go/sdk/azidentity"
"github.com/Azure/azure-sdk-for-go/sdk/storage/azblob/service"
serviceBfs "github.com/Azure/azure-sdk-for-go/sdk/storage/azdatalake/service"
"github.com/Azure/azure-storage-fuse/v2/common/log"
)

// Verify that the Auth implement the correct AzAuth interfaces
var _ azAuth = &azAuthBlobClientAssertion{}
var _ azAuth = &azAuthDatalakeClientAssertion{}

type azAuthClientAssertion struct {
azAuthBase
azOAuthBase
}

func (azclientassertion *azAuthClientAssertion) getTokenCredential() (azcore.TokenCredential, error) {
opts := azclientassertion.getAzIdentityClientOptions(&azclientassertion.config)

// Create MSI cred to fetch token
msiOpts := &azidentity.ManagedIdentityCredentialOptions{
ClientOptions: opts,
}
msiOpts.ID = azidentity.ClientID(azclientassertion.config.ApplicationID)
cred, err := azidentity.NewManagedIdentityCredential(msiOpts)
if err != nil {
log.Err("azAuthClientAssertion::getTokenCredential : Failed to create managed identity credential [%s]", err.Error())
return nil, err
}

scope := "api://AzureADTokenExchange"
if azclientassertion.config.AuthResource != "" {
scope = azclientassertion.config.AuthResource
}

getClientAssertions := func(context.Context) (string, error) {
token, err := cred.GetToken(context.Background(), policy.TokenRequestOptions{
Scopes: []string{scope},
})

if err != nil {
log.Err("azAuthClientAssertion::getTokenCredential : Failed to get token from managed identity credential [%s]", err.Error())
return "", err
}

return token.Token, nil
}

if azclientassertion.config.UserAssertion == "" {
assertOpts := &azidentity.ClientAssertionCredentialOptions{
ClientOptions: opts,
}

return azidentity.NewClientAssertionCredential(
azclientassertion.config.TenantID,
azclientassertion.config.ClientID,
getClientAssertions,
assertOpts)
} else {
assertOpts := &azidentity.OnBehalfOfCredentialOptions{
ClientOptions: opts,
}

return azidentity.NewOnBehalfOfCredentialWithClientAssertions(
azclientassertion.config.TenantID,
azclientassertion.config.ClientID,
azclientassertion.config.UserAssertion,
getClientAssertions,
assertOpts)
}
}

type azAuthBlobClientAssertion struct {
azAuthClientAssertion
}

// getServiceClient : returns SPN based service client for blob
func (azclientassertion *azAuthBlobClientAssertion) getServiceClient(stConfig *AzStorageConfig) (interface{}, error) {
cred, err := azclientassertion.getTokenCredential()
if err != nil {
log.Err("azAuthBlobClientAssertion::getServiceClient : Failed to get token credential from client assertion [%s]", err.Error())
return nil, err
}

opts, err := getAzBlobServiceClientOptions(stConfig)
if err != nil {
log.Err("azAuthBlobClientAssertion::getServiceClient : Failed to create client options [%s]", err.Error())
return nil, err
}

svcClient, err := service.NewClient(azclientassertion.config.Endpoint, cred, opts)
if err != nil {
log.Err("azAuthBlobClientAssertion::getServiceClient : Failed to create service client [%s]", err.Error())
}

return svcClient, err
}

type azAuthDatalakeClientAssertion struct {
azAuthClientAssertion
}

// getServiceClient : returns SPN based service client for blob
func (azclientassertion *azAuthDatalakeClientAssertion) getServiceClient(stConfig *AzStorageConfig) (interface{}, error) {
cred, err := azclientassertion.getTokenCredential()
if err != nil {
log.Err("azAuthDatalakeClientAssertion::getServiceClient : Failed to get token credential from client assertion [%s]", err.Error())
return nil, err
}

opts, err := getAzDatalakeServiceClientOptions(stConfig)
if err != nil {
log.Err("azAuthDatalakeClientAssertion::getServiceClient : Failed to create client options [%s]", err.Error())
return nil, err
}

svcClient, err := serviceBfs.NewClient(azclientassertion.config.Endpoint, cred, opts)
if err != nil {
log.Err("azAuthDatalakeClientAssertion::getServiceClient : Failed to create service client [%s]", err.Error())
}

return svcClient, err
}
17 changes: 17 additions & 0 deletions component/azstorage/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,10 @@ func (AuthType) AZCLI() AuthType {
return AuthType(5)
}

func (AuthType) CLIENTASSERTION() AuthType {
return AuthType(6)
}

func (a AuthType) String() string {
return enum.StringInt(a, reflect.TypeOf(a))
}
Expand Down Expand Up @@ -144,6 +148,7 @@ const (
EnvAzAuthResource = "AZURE_STORAGE_AUTH_RESOURCE"
EnvAzStorageCpkEncryptionKey = "AZURE_STORAGE_CPK_ENCRYPTION_KEY"
EnvAzStorageCpkEncryptionKeySha256 = "AZURE_STORAGE_CPK_ENCRYPTION_KEY_SHA256"
EnvAzUserAssertion = "AZURE_STORAGE_USER_ASSERTION"
)

type AzStorageOptions struct {
Expand Down Expand Up @@ -189,6 +194,7 @@ type AzStorageOptions struct {
CPKEncryptionKeySha256 string `config:"cpk-encryption-key-sha256" yaml:"cpk-encryption-key-sha256"`
PreserveACL bool `config:"preserve-acl" yaml:"preserve-acl"`
Filter string `config:"filter" yaml:"filter"`
UserAssertion string `config:"user-assertion" yaml:"user-assertions"`

// v1 support
UseAdls bool `config:"use-adls" yaml:"-"`
Expand Down Expand Up @@ -230,6 +236,7 @@ func RegisterEnvVariables() {
config.BindEnv("azstorage.cpk-encryption-key", EnvAzStorageCpkEncryptionKey)
config.BindEnv("azstorage.cpk-encryption-key-sha256", EnvAzStorageCpkEncryptionKeySha256)

config.BindEnv("azstorage.user-assertion", EnvAzUserAssertion)
}

// ----------- Config Parsing and Validation ---------------
Expand Down Expand Up @@ -467,6 +474,16 @@ func ParseAndValidateConfig(az *AzStorage, opt AzStorageOptions) error {
az.stConfig.authConfig.WorkloadIdentityToken = opt.WorkloadIdentityToken
case EAuthType.AZCLI():
az.stConfig.authConfig.AuthMode = EAuthType.AZCLI()
case EAuthType.CLIENTASSERTION():
az.stConfig.authConfig.AuthMode = EAuthType.CLIENTASSERTION()
if opt.ClientID == "" || opt.TenantID == "" || opt.ApplicationID == "" {
return errors.New("Client ID, Tenant ID or Application ID not provided")
}

az.stConfig.authConfig.ClientID = opt.ClientID
az.stConfig.authConfig.TenantID = opt.TenantID
az.stConfig.authConfig.ApplicationID = opt.ApplicationID
az.stConfig.authConfig.UserAssertion = opt.UserAssertion

default:
log.Err("ParseAndValidateConfig : Invalid auth mode %s", opt.AuthMode)
Expand Down
Loading