[BUG] Azure.Identity DefaultAzureCredential in Visual Studio fails when MFA required

[Jawvig]

Describe the bug
From within Visual Studio, running code that uses DefaultAzureCredential with an account that requires MFA results in an exception. The credential was used with a BlobContainerClient from the v12 Azure Storage client library. The problem can be reproduced in a Console app running in Debug in Visual Studio but also occurs when using MS Test or ReSharper test runners.
Visual Studio contains the required account in the list of accounts (Account Settings -> All Accounts) and the account was removed and re-added (and thus re-authenticated).

Expected behavior
The DefaultAzureCredential should either just work or move into an OAuth workflow to obtain a token.

Actual behavior (include Exception or Stack Trace)
Exception thrown at the point the first data is attempted to be retrieved:

Azure.Identity.AuthenticationFailedException : DefaultAzureCredential authentication failed.
---- Azure.Identity.AuthenticationFailedException : VisualStudioCodeCredential authentication failed.
-------- Microsoft.Identity.Client.MsalUiRequiredException : AADSTS50076: Due to a configuration change made by your administrator, or because you moved to a new location, you must use multi-factor authentication to access 'xxxxxxxx'.
Trace ID: xxxxx
Correlation ID: xxxxx
Timestamp: 2020-04-25 10:43:02Z
   at Azure.Identity.DefaultAzureCredential.GetTokenImplAsync(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken)
   at Azure.Identity.DefaultAzureCredential.GetTokenAsync(TokenRequestContext requestContext, CancellationToken cancellationToken)
   at Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
   at Azure.Core.Pipeline.HttpPipelineSynchronousPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline)
   at Azure.Core.Pipeline.RetryPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
   at Azure.Core.Pipeline.RetryPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
   at Azure.Core.Pipeline.HttpPipelineSynchronousPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline)
   at Azure.Core.Pipeline.HttpPipelineSynchronousPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline)
   at Azure.Storage.Blobs.BlobRestClient.Container.ListBlobsFlatSegmentAsync(ClientDiagnostics clientDiagnostics, HttpPipeline pipeline, Uri resourceUri, String version, String prefix, String marker, Nullable`1 maxresults, IEnumerable`1 include, Nullable`1 timeout, String requestId, Boolean async, String operationName, CancellationToken cancellationToken)
   at Azure.Storage.Blobs.BlobContainerClient.GetBlobsInternal(String marker, BlobTraits traits, BlobStates states, String prefix, Nullable`1 pageSizeHint, Boolean async, CancellationToken cancellationToken)
   at Azure.Storage.Blobs.Models.GetBlobsAsyncCollection.GetNextPageAsync(String continuationToken, Nullable`1 pageSizeHint, Boolean async, CancellationToken cancellationToken)
   at Azure.Storage.StorageCollectionEnumerator`1.StorageAsyncPageable.GetAsyncEnumerator(CancellationToken cancellationToken)+MoveNext()
   at Azure.Storage.StorageCollectionEnumerator`1.StorageAsyncPageable.GetAsyncEnumerator(CancellationToken cancellationToken)+System.Threading.Tasks.Sources.IValueTaskSource<System.Boolean>.GetResult()
   at Matchnet.LabelledModelPopulator.LabellingProjects.LabellingProjectRepository.GetRawProjects()+MoveNext() in C:\Users\josh_\source\repos\matchnet\email-pdf-matching\Matchnet.LabelledModelPopulator\LabellingProjects\LabellingProjectRepository.cs:line 65
   at Matchnet.LabelledModelPopulator.LabellingProjects.LabellingProjectRepository.GetRawProjects()+MoveNext() in C:\Users\josh_\source\repos\matchnet\email-pdf-matching\Matchnet.LabelledModelPopulator\LabellingProjects\LabellingProjectRepository.cs:line 65
   at Matchnet.LabelledModelPopulator.LabellingProjects.LabellingProjectRepository.GetRawProjects()+System.Threading.Tasks.Sources.IValueTaskSource<System.Boolean>.GetResult()
   at System.Linq.AsyncEnumerable.SelectEnumerableAsyncIterator`2.MoveNextCore() in /_/Ix.NET/Source/System.Linq.Async/System/Linq/Operators/Select.cs:line 199
   at System.Linq.AsyncIteratorBase`1.MoveNextAsync() in /_/Ix.NET/Source/System.Linq.Async/System/Linq/AsyncIterator.cs:line 70
   at System.Linq.AsyncIteratorBase`1.MoveNextAsync() in /_/Ix.NET/Source/System.Linq.Async/System/Linq/AsyncIterator.cs:line 75
   at System.Linq.AsyncEnumerable.<ToListAsync>g__Core|620_0[TSource](IAsyncEnumerable`1 source, CancellationToken cancellationToken) in /_/Ix.NET/Source/System.Linq.Async/System/Linq/Operators/ToList.cs:line 36
   at System.Linq.AsyncEnumerable.<ToListAsync>g__Core|620_0[TSource](IAsyncEnumerable`1 source, CancellationToken cancellationToken) in /_/Ix.NET/Source/System.Linq.Async/System/Linq/Operators/ToList.cs:line 36
   at  *CLIENT CODE omitted*
--- End of stack trace from previous location where exception was thrown ---
----- Inner Stack Trace -----
   at Azure.Identity.VisualStudioCodeCredential.GetTokenImplAsync(TokenRequestContext requestContext, Boolean async, CancellationToken cancellationToken)
   at Azure.Identity.VisualStudioCodeCredential.GetTokenAsync(TokenRequestContext requestContext, CancellationToken cancellationToken)
   at Azure.Identity.DefaultAzureCredential.GetTokenFromSourcesAsync(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken)
   at Azure.Identity.DefaultAzureCredential.GetTokenImplAsync(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken)
----- Inner Stack Trace -----
   at Microsoft.Identity.Client.OAuth2.OAuth2Client.CreateErrorResponse(HttpResponse response, RequestContext requestContext)
   at Microsoft.Identity.Client.OAuth2.OAuth2Client.CreateResponse[T](HttpResponse response, RequestContext requestContext, Boolean addCorrelationId)
   at Microsoft.Identity.Client.OAuth2.OAuth2Client.ExecuteRequestAsync[T](Uri endPoint, HttpMethod method, RequestContext requestContext)
   at Microsoft.Identity.Client.OAuth2.OAuth2Client.GetTokenAsync(Uri endPoint, RequestContext requestContext)
   at Microsoft.Identity.Client.Internal.Requests.RequestBase.SendHttpMessageAsync(OAuth2Client client, String tokenEndpoint)
   at Microsoft.Identity.Client.Internal.Requests.RequestBase.SendTokenRequestAsync(String tokenEndpoint, IDictionary`2 additionalBodyParameters, CancellationToken cancellationToken)
   at Microsoft.Identity.Client.Internal.Requests.ByRefreshTokenRequest.ExecuteAsync(CancellationToken cancellationToken)
   at Microsoft.Identity.Client.Internal.Requests.RequestBase.RunAsync(CancellationToken cancellationToken)
   at Microsoft.Identity.Client.ApiConfig.Executors.ClientApplicationBaseExecutor.ExecuteAsync(AcquireTokenCommonParameters commonParameters, AcquireTokenByRefreshTokenParameters refreshTokenParameters, CancellationToken cancellationToken)
   at Azure.Identity.AbstractAcquireTokenParameterBuilderExtensions.ExecuteAsync[T](AbstractAcquireTokenParameterBuilder`1 builder, Boolean async, CancellationToken cancellationToken)
   at Azure.Identity.MsalPublicClient.AcquireTokenWithDeviceCodeAsync(String[] scopes, String storedCredentials, AzureCloudInstance azureCloudInstance, String tenant, Boolean async, CancellationToken cancellationToken)
   at Azure.Identity.VisualStudioCodeCredential.GetTokenImplAsync(TokenRequestContext requestContext, Boolean async, CancellationToken cancellationToken)

To Reproduce
Steps to reproduce the behavior (include a code snippet, screenshot, or any additional information that might help us reproduce the issue)

var blobContainerUri = new Uri($"https://{StorageAccount}.blob.core.windows.net/{ContainerName}");
var blobContainerClient = new BlobContainerClient(blobContainerUri, new DefaultAzureCredential());
await foreach (var blob in blobContainerClient.GetBlobsAsync())
{
  Console.WriteLine(blob.Name);
}

Environment:

• Azure.Storage.Blobs 12.4.1
• Azure.Identity 1.2.0-preview.2
• Visual Studio Community 2019 Preview Version 16.6.0 Preview 4.0
• donet --info output:

.NET Core SDK (reflecting any global.json):
Version:   3.1.300-preview-015115
Commit:    cd8d468337
Runtime Environment:
OS Name:     Windows
OS Version:  10.0.19613
OS Platform: Windows
RID:         win10-x64
Base Path:   C:\Program Files\dotnet\sdk\3.1.300-preview-015115\
Host (useful for support):
Version: 3.1.3
Commit:  4a9f85e9f8
.NET Core SDKs installed:
2.1.701 [C:\Program Files\dotnet\sdk]
2.1.801 [C:\Program Files\dotnet\sdk]
2.2.301 [C:\Program Files\dotnet\sdk]
2.2.401 [C:\Program Files\dotnet\sdk]
3.1.201 [C:\Program Files\dotnet\sdk]
3.1.300-preview-015115 [C:\Program Files\dotnet\sdk]

[Jawvig]

I should also note that using new DefaultAzureCredential(true), to attempt to open an interactive browser, does not result in any different behaviour.

[AlexanderSher]

Hi @Jawvig ,
Inner stack trace shows failing VisualStudioCodeCredential. Do you have your account in both Visual Studio and Visual Studio Code?

[Jawvig]

Ah, I had missed the "Code" part of the VisualStudioCodeCredential. I probably did register that account in VS Code before the MFA requirement was added. After going into VS Code, opening "Azure Storage" in the explorer and signing in, the code above works.

So, I'm sorted out. However, should it have fallen back on some other mechanism when that one failed? Visual Studio or Azure CLI?

[AlexanderSher]

Visual Studio and Azure CLI are checked first, so they have failed for some reason. However, VisualStudioCodeCredential must handle MsalUiRequiredException, so that's a bug by itself.

[Jawvig]

Is the VisualStudioCodeCredential the last in the chain, hence why its failure is the one getting bubbled up? Not quite sure where to take this from here. Re-name this bug to "VisualStudioCodeCredential must handle MsalUiRequiredException" or dig further into why the Visual Studio and Azure CLI failed?

[AlexanderSher]

Both. Can I ask you to sign out from Visual Studio Code Azure Account and try to reproduce this bug again?

[Jawvig]

Sure. I ran the "Azure: Sign Out" command in VS Code and re-ran the sample in debug from Visual Studio. I got the following exception. I closed both and reopened Visual Studio to run it again but with the same result.

Azure.Identity.AuthenticationFailedException: DefaultAzureCredential authentication failed.
 ---> Azure.Identity.AuthenticationFailedException: VisualStudioCodeCredential authentication failed.
 ---> System.InvalidOperationException: CredRead has failed but error is unknown.
   at Azure.Identity.WindowsNativeMethods.ThrowIfFailed(Boolean isSucceeded, String methodName)
   at Azure.Identity.WindowsNativeMethods.CredRead(String target, CRED_TYPE type)
   at Azure.Identity.WindowsVisualStudioCodeAdapter.GetCredentials(String serviceName, String accountName)
   at Azure.Identity.VisualStudioCodeCredential.GetTokenImplAsync(TokenRequestContext requestContext, Boolean async, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at Azure.Identity.VisualStudioCodeCredential.GetTokenImplAsync(TokenRequestContext requestContext, Boolean async, CancellationToken cancellationToken)
   at Azure.Identity.VisualStudioCodeCredential.GetTokenAsync(TokenRequestContext requestContext, CancellationToken cancellationToken)
   at Azure.Identity.DefaultAzureCredential.GetTokenFromSourcesAsync(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken)
   at Azure.Identity.DefaultAzureCredential.GetTokenImplAsync(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at Azure.Identity.DefaultAzureCredential.GetTokenImplAsync(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken)
   at Azure.Identity.DefaultAzureCredential.GetTokenAsync(TokenRequestContext requestContext, CancellationToken cancellationToken)
   at Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
   at Azure.Core.Pipeline.HttpPipelineSynchronousPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline)
   at Azure.Core.Pipeline.RetryPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
   at Azure.Core.Pipeline.RetryPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
   at Azure.Core.Pipeline.HttpPipelineSynchronousPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline)
   at Azure.Core.Pipeline.HttpPipelineSynchronousPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline)
   at Azure.Storage.Blobs.BlobRestClient.Container.ListBlobsFlatSegmentAsync(ClientDiagnostics clientDiagnostics, HttpPipeline pipeline, Uri resourceUri, String version, String prefix, String marker, Nullable`1 maxresults, IEnumerable`1 include, Nullable`1 timeout, String requestId, Boolean async, String operationName, CancellationToken cancellationToken)
   at Azure.Storage.Blobs.BlobContainerClient.GetBlobsInternal(String marker, BlobTraits traits, BlobStates states, String prefix, Nullable`1 pageSizeHint, Boolean async, CancellationToken cancellationToken)
   at Azure.Storage.Blobs.Models.GetBlobsAsyncCollection.GetNextPageAsync(String continuationToken, Nullable`1 pageSizeHint, Boolean async, CancellationToken cancellationToken)
   at Azure.Storage.StorageCollectionEnumerator`1.StorageAsyncPageable.GetAsyncEnumerator(CancellationToken cancellationToken)+MoveNext()
   at Azure.Storage.StorageCollectionEnumerator`1.StorageAsyncPageable.GetAsyncEnumerator(CancellationToken cancellationToken)+System.Threading.Tasks.Sources.IValueTaskSource<System.Boolean>.GetResult()
   at DefaultAzureCredentialMfaRepro.Program.Main(String[] args) in C:\Users\josh_\source\repos\DefaultAzureCredentialMfaRepro\Program.cs:line 17
   at DefaultAzureCredentialMfaRepro.Program.Main(String[] args) in C:\Users\josh_\source\repos\DefaultAzureCredentialMfaRepro\Program.cs:line 17

[AlexanderSher]

Ok, let's try to disable VisualStudioCodeCredential explicitly. In your sample, can you change the line that creates blobContainerClient to

var credential = new DefaultAzureCredential(new DefaultAzureCredentialOptions() {ExcludeVisualStudioCodeCredential = true});
var blobContainerClient = new BlobContainerClient(blobContainerUri, credential);

[Jawvig]

I've done that and this time it works without error.
This bug title seems off the mark at this point. 😀

[AlexanderSher]

Ok, then we have two bugs related to VisualStudioCodeCredential. Thank you for your help :)

[xperiandri]

Tried 1.2.0-preview.1-1.2.0-preview.5 works only with interactive auth and AZURE_USERNAME and AZURE_TENANT_ID

var credentialOptions = new DefaultAzureCredentialOptions
{
    ExcludeInteractiveBrowserCredential = false,
    ExcludeVisualStudioCodeCredential = true
};
var credential = new DefaultAzureCredential(credentialOptions);

If ExcludeVisualStudioCodeCredential = false then exception.
If ExcludeInteractiveBrowserCredential = true and ExcludeVisualStudioCodeCredential = true then cannot find my Microsoft Account in cache.