dotnet format throws because it can't determine file encodings in 9.0.200

[DavidZidar]

Describe the bug

It is no longer possible to use dotnet format without arguments to format code in certain projects because it has started throwing an error in 9.0.200. dotnet format --verify-no-changes does still work however.

What's worse, it doesn't specify what file might be the issue so it is very hard to figure out what might be going wrong.

To Reproduce

I'm not exactly sure since it doesn't tell me what file it has a problem with, but I'm just running dotnet format in an ASP.NET Core 9 project with about 1500 C# files.

Exceptions (if any)

Unhandled exception: System.Exception: source text did not have an identifiable encoding
   at Microsoft.CodeAnalysis.Tools.Formatters.CharsetFormatter.IsEncodingEquivalent(SourceText sourceText, Encoding encoding)
   at Microsoft.CodeAnalysis.Tools.Formatters.CharsetFormatter.<>c__DisplayClass10_0.<FormatFileAsync>b__0()
   at System.Threading.Tasks.Task`1.InnerInvoke()
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location ---
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread threadPoolThread)
--- End of stack trace from previous location ---
   at Microsoft.CodeAnalysis.Tools.Formatters.DocumentFormatter.GetFormattedSourceTextAsync(Document document, OptionSet optionSet, AnalyzerConfigOptions analyzerConfigOptions, FormatOptions formatOptions, ILogger logger, CancellationToken cancellationToken)
   at Microsoft.CodeAnalysis.Tools.Formatters.DocumentFormatter.<>c__DisplayClass8_1.<<FormatFiles>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at Microsoft.CodeAnalysis.Tools.Formatters.DocumentFormatter.ApplyFileChangesAsync(Solution solution, ImmutableArray`1 formattedDocuments, FormatOptions formatOptions, ILogger logger, List`1 formattedFiles, CancellationToken cancellationToken)
   at Microsoft.CodeAnalysis.Tools.Formatters.DocumentFormatter.FormatAsync(Workspace workspace, Solution solution, ImmutableArray`1 formattableDocuments, FormatOptions formatOptions, ILogger logger, List`1 formattedFiles, CancellationToken cancellationToken)
   at Microsoft.CodeAnalysis.Tools.CodeFormatter.RunCodeFormattersAsync(Workspace workspace, Solution solution, ImmutableArray`1 formattableDocuments, FormatOptions formatOptions, ILogger logger, List`1 formattedFiles, CancellationToken cancellationToken)
   at Microsoft.CodeAnalysis.Tools.CodeFormatter.FormatWorkspaceAsync(FormatOptions formatOptions, ILogger logger, CancellationToken cancellationToken, String binaryLogPath)
   at Microsoft.CodeAnalysis.Tools.FormatCommandCommon.FormatAsync(FormatOptions formatOptions, ILogger`1 logger, CancellationToken cancellationToken)
   at Microsoft.CodeAnalysis.Tools.Commands.RootFormatCommand.FormatCommandDefaultHandler.InvokeAsync(ParseResult parseResult, CancellationToken cancellationToken)
   at System.CommandLine.Invocation.InvocationPipeline.InvokeAsync(ParseResult parseResult, CancellationToken cancellationToken)

Further technical details

The error started showing up after installing Visual Studio 17.3, but it is also happening in WSL2 with the latest .NET SDK.

[DavidZidar]

I did try running dotnet format --verbosity diag but did not receive any additional actionable information.

[adam-knights]

Incase its useful in fixing this, we see similar today having taken 9.0.200, but on Windows only, our Mac arm64 users do not see the errors. May not apply to all cases, but if someone is looking at this and can't reproduce, try Windows.

[caleblloyd]

I am hitting this on linux after the 9.0.200 upgrade today. Same deal, --verbosity diag doesn't show any other useful information. Reverting to 9.0.103 fixes.

[ChrisTTian667]

Incase its useful in fixing this, we see similar today having taken 9.0.200, but on Windows only, our Mac arm64 users do not see the errors. May not apply to all cases, but if someone is looking at this and can't reproduce, try Windows.

I'm a Mac ARM 64 User and have the same issue. I don't think that its related to the platform.

[MattEdwardsWaggleBee]

I'm seeing this on my Windows 11 machine and my GitHub action running on "ubuntu-latest"

update: I'm on 9.0.200 co-worker doesn't see this on 9.0.103