Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve processing in FindRefs #73253

Merged
merged 47 commits into from
Apr 28, 2024
Merged
Show file tree
Hide file tree
Changes from 25 commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
d8d1b78
REmove unused part of FAR api
CyrusNajmabadi Apr 27, 2024
ecd1f39
parallel
CyrusNajmabadi Apr 27, 2024
94c0662
Docs
CyrusNajmabadi Apr 27, 2024
1042c63
in progrss
CyrusNajmabadi Apr 27, 2024
269b8a3
in progress
CyrusNajmabadi Apr 27, 2024
cee9783
No scratch buffer
CyrusNajmabadi Apr 27, 2024
6e1adc1
Cache data up front
CyrusNajmabadi Apr 27, 2024
f17f727
simplify
CyrusNajmabadi Apr 27, 2024
57c3ae6
Simplify
CyrusNajmabadi Apr 27, 2024
2dedcbe
IN progress
CyrusNajmabadi Apr 27, 2024
daa291e
IN progress
CyrusNajmabadi Apr 27, 2024
3daa54f
IN progress
CyrusNajmabadi Apr 27, 2024
1e351f5
Prime cache
CyrusNajmabadi Apr 27, 2024
4a6d2cb
Simpliify
CyrusNajmabadi Apr 27, 2024
de5f7e5
in progress
CyrusNajmabadi Apr 27, 2024
c8c6b37
Working
CyrusNajmabadi Apr 27, 2024
99a8b91
in progress
CyrusNajmabadi Apr 27, 2024
6679c60
in progress
CyrusNajmabadi Apr 27, 2024
cc563f7
in progress
CyrusNajmabadi Apr 27, 2024
293f1ba
in progress
CyrusNajmabadi Apr 27, 2024
e8f85f9
in progress
CyrusNajmabadi Apr 27, 2024
40c0f65
in progress
CyrusNajmabadi Apr 27, 2024
bab6cf3
in progress
CyrusNajmabadi Apr 27, 2024
7917020
All sync
CyrusNajmabadi Apr 27, 2024
751af97
Passing
CyrusNajmabadi Apr 27, 2024
753ae44
spelling'
CyrusNajmabadi Apr 27, 2024
d1689c3
Delete
CyrusNajmabadi Apr 27, 2024
91465a7
Strong refs
CyrusNajmabadi Apr 27, 2024
98f6cd7
Simplify
CyrusNajmabadi Apr 27, 2024
8a43b5c
Use same helper
CyrusNajmabadi Apr 27, 2024
22b2dff
Merge remote-tracking branch 'upstream/main' into navToCleanup
CyrusNajmabadi Apr 27, 2024
b99480c
Merge branch 'navToCleanup' into farParallel
CyrusNajmabadi Apr 27, 2024
298b1ec
share code
CyrusNajmabadi Apr 27, 2024
03b0fe7
move outside loop
CyrusNajmabadi Apr 27, 2024
5bcb1ca
revert
CyrusNajmabadi Apr 27, 2024
3bcce89
revert
CyrusNajmabadi Apr 27, 2024
54639c6
pull notifications out
CyrusNajmabadi Apr 27, 2024
2f64957
Fix
CyrusNajmabadi Apr 28, 2024
df96626
Make normal methods
CyrusNajmabadi Apr 28, 2024
13b1f1c
Spelling
CyrusNajmabadi Apr 28, 2024
3ae4a71
Simplify
CyrusNajmabadi Apr 28, 2024
432b3fd
remove
CyrusNajmabadi Apr 28, 2024
dc8b00d
Rename file
CyrusNajmabadi Apr 28, 2024
d30867d
Equals
CyrusNajmabadi Apr 28, 2024
0a32ab9
Equals
CyrusNajmabadi Apr 28, 2024
48fdc8e
restore
CyrusNajmabadi Apr 28, 2024
176e239
Work
CyrusNajmabadi Apr 28, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -58,14 +58,6 @@ public void OnCompleted()
{
}

public void OnFindInDocumentStarted(Document document)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FAR had a system to notify consumers "i'm startign a doc" and "i'm done with a doc". The original thinking was to use that for progress, to display which file we're looking at.

but:

  1. we never hooked this up to naything at all.
  2. this wouldn't be a good experience anyways, since we're extremely parallel. so this was removed entirely.

{
}

public void OnFindInDocumentCompleted(Document document)
{
}

private static bool FilterDefinition(ISymbol definition)
{
return definition.IsImplicitlyDeclared ||
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
#nullable disable

using System.Collections.Generic;
using System.Collections.Immutable;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.CodeAnalysis.Classification;
Expand Down Expand Up @@ -73,8 +74,6 @@ public IStreamingProgressTracker ProgressTracker
// any of these.
public ValueTask OnStartedAsync(CancellationToken cancellationToken) => default;
public ValueTask OnCompletedAsync(CancellationToken cancellationToken) => default;
public ValueTask OnFindInDocumentStartedAsync(Document document, CancellationToken cancellationToken) => default;
public ValueTask OnFindInDocumentCompletedAsync(Document document, CancellationToken cancellationToken) => default;
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FAR had a system to notify consumers "i'm startign a doc" and "i'm done with a doc". The original thinking was to use that for progress, to display which file we're looking at.

but:

we never hooked this up to naything at all.
this wouldn't be a good experience anyways, since we're extremely parallel. so this was removed entirely.


// More complicated forwarding functions. These need to map from the symbols
// used by the FAR engine to the INavigableItems used by the streaming FAR
Expand Down Expand Up @@ -107,17 +106,21 @@ public async ValueTask OnDefinitionFoundAsync(SymbolGroup group, CancellationTok
await context.OnDefinitionFoundAsync(definitionItem, cancellationToken).ConfigureAwait(false);
}

public async ValueTask OnReferenceFoundAsync(SymbolGroup group, ISymbol definition, ReferenceLocation location, CancellationToken cancellationToken)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

view with whitespcae off.

public async ValueTask OnReferencesFoundAsync(
ImmutableArray<(SymbolGroup group, ISymbol symbol, ReferenceLocation location)> references, CancellationToken cancellationToken)
{
var definitionItem = await GetDefinitionItemAsync(group, cancellationToken).ConfigureAwait(false);
var referenceItem = await location.TryCreateSourceReferenceItemAsync(
classificationOptions,
definitionItem,
includeHiddenLocations: false,
cancellationToken).ConfigureAwait(false);

if (referenceItem != null)
await context.OnReferenceFoundAsync(referenceItem, cancellationToken).ConfigureAwait(false);
foreach (var (group, _, location) in references)
{
var definitionItem = await GetDefinitionItemAsync(group, cancellationToken).ConfigureAwait(false);
var referenceItem = await location.TryCreateSourceReferenceItemAsync(
classificationOptions,
definitionItem,
includeHiddenLocations: false,
cancellationToken).ConfigureAwait(false);

if (referenceItem != null)
await context.OnReferenceFoundAsync(referenceItem, cancellationToken).ConfigureAwait(false);
}
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ await context.ReportNoResultsAsync(
var (symbol, project) = symbolAndProjectOpt.Value;

var solution = project.Solution;
var bases = await FindBaseHelpers.FindBasesAsync(symbol, solution, cancellationToken).ConfigureAwait(false);
var bases = FindBaseHelpers.FindBases(symbol, solution, cancellationToken);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a lot of FAR and helpers becamse sync. will explain how later in pr.

if (bases.Length == 0 && symbol is IMethodSymbol { MethodKind: MethodKind.Constructor } constructor)
{
var nextConstructor = await FindNextConstructorInChainAsync(solution, constructor, cancellationToken).ConfigureAwait(false);
Expand Down
14 changes: 5 additions & 9 deletions src/Features/Core/Portable/GoToBase/FindBaseHelpers.cs
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ namespace Microsoft.CodeAnalysis.GoToBase;

internal static class FindBaseHelpers
{
public static ValueTask<ImmutableArray<ISymbol>> FindBasesAsync(
public static ImmutableArray<ISymbol> FindBases(
ISymbol symbol, Solution solution, CancellationToken cancellationToken)
{
if (symbol is INamedTypeSymbol
Expand All @@ -23,16 +23,12 @@ public static ValueTask<ImmutableArray<ISymbol>> FindBasesAsync(
} namedTypeSymbol)
{
var result = BaseTypeFinder.FindBaseTypesAndInterfaces(namedTypeSymbol).CastArray<ISymbol>();
return ValueTaskFactory.FromResult(result);
return result;
}

if (symbol.Kind is SymbolKind.Property or
SymbolKind.Method or
SymbolKind.Event)
{
return BaseTypeFinder.FindOverriddenAndImplementedMembersAsync(symbol, solution, cancellationToken);
}
if (symbol.Kind is SymbolKind.Property or SymbolKind.Method or SymbolKind.Event)
return BaseTypeFinder.FindOverriddenAndImplementedMembers(symbol, solution, cancellationToken);

return ValueTaskFactory.FromResult(ImmutableArray<ISymbol>.Empty);
return [];
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
// The .NET Foundation licenses this file to you under the MIT license.
// See the LICENSE file in the project root for more information.

using System.Collections.Immutable;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.CodeAnalysis.FindSymbols;
Expand All @@ -28,11 +29,16 @@ private class FindReferencesProgress(OperationCollector valueTrackingProgressCol

public ValueTask OnDefinitionFoundAsync(SymbolGroup symbolGroup, CancellationToken _) => new();

public ValueTask OnFindInDocumentCompletedAsync(Document document, CancellationToken _) => new();

public ValueTask OnFindInDocumentStartedAsync(Document document, CancellationToken _) => new();
public async ValueTask OnReferencesFoundAsync(
ImmutableArray<(SymbolGroup group, ISymbol symbol, ReferenceLocation location)> references,
CancellationToken cancellationToken)
{
foreach (var (_, symbol, location) in references)
await OnReferenceFoundAsync(symbol, location, cancellationToken).ConfigureAwait(false);
}

public async ValueTask OnReferenceFoundAsync(SymbolGroup _, ISymbol symbol, ReferenceLocation location, CancellationToken cancellationToken)
private async ValueTask OnReferenceFoundAsync(
ISymbol symbol, ReferenceLocation location, CancellationToken cancellationToken)
{
if (!location.Location.IsInSource)
{
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,10 +15,10 @@ internal static partial class BaseTypeFinder
public static ImmutableArray<INamedTypeSymbol> FindBaseTypesAndInterfaces(INamedTypeSymbol type)
=> FindBaseTypes(type).AddRange(type.AllInterfaces);

public static async ValueTask<ImmutableArray<ISymbol>> FindOverriddenAndImplementedMembersAsync(
public static ImmutableArray<ISymbol> FindOverriddenAndImplementedMembers(
ISymbol symbol, Solution solution, CancellationToken cancellationToken)
{
var results = ArrayBuilder<ISymbol>.GetInstance();
using var _ = ArrayBuilder<ISymbol>.GetInstance(out var results);

// This is called for all: class, struct or interface member.
results.AddRange(symbol.ExplicitOrImplicitInterfaceImplementations());
Expand All @@ -31,7 +31,7 @@ public static async ValueTask<ImmutableArray<ISymbol>> FindOverriddenAndImplemen
cancellationToken.ThrowIfCancellationRequested();

// Add to results overridden members only. Do not add hidden members.
if (await SymbolFinder.IsOverrideAsync(solution, symbol, member, cancellationToken).ConfigureAwait(false))
if (SymbolFinder.IsOverride(solution, symbol, member))
{
results.Add(member);

Expand Down Expand Up @@ -64,7 +64,8 @@ public static async ValueTask<ImmutableArray<ISymbol>> FindOverriddenAndImplemen
}

// Remove duplicates from interface implementations before adding their projects.
return results.ToImmutableAndFree().Distinct();
results.RemoveDuplicates();
return results.ToImmutableAndClear();
}

private static ImmutableArray<INamedTypeSymbol> FindBaseTypes(INamedTypeSymbol type)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,29 +32,44 @@ public static async ValueTask<FindReferenceCache> GetCacheAsync(Document documen

static async Task<FindReferenceCache> ComputeCacheAsync(Document document, CancellationToken cancellationToken)
{
var text = await document.GetValueTextAsync(cancellationToken).ConfigureAwait(false);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the cache is an object we make for a document once we've decided it's a doc of interest and we're going ot be searching it. that means it already registered a 'hit' during the pass when we asked to find docments that could potnetially contain the symbols we're searching for (say because it got a hit in the bloomfilter of all the names in the doc).

Once we've decided we're looking at the doc, it makes sense to get all the info we need up front, so that hte search in the doc doesn't ever ahve to go async. So we do this here. Note: teh txt is the most normal thing we need in a normal find refs. Say we're lookign for Foo and we have a hit in a doc. We actually use the text to find the positions in the text to try to find tokens to actually bind. So really, there's no benefit deferring these things since we will practically always need them.


// Find-Refs is not impacted by nullable types at all. So get a nullable-disabled semantic model to avoid
// unnecessary costs while binding.
var model = await document.GetRequiredNullableDisabledSemanticModelAsync(cancellationToken).ConfigureAwait(false);
var root = await document.GetRequiredSyntaxRootAsync(cancellationToken).ConfigureAwait(false);
return new(document, model, root);

// It's very costly to walk an entire tree. So if the tree is simple and doesn't contain
// any unicode escapes in it, then we do simple string matching to find the tokens.
var index = await SyntaxTreeIndex.GetRequiredIndexAsync(document, cancellationToken).ConfigureAwait(false);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

similarly, we will have practically always computed thsi for all documents that were hits anyways (because that's how we figured out there might be a hit in the first place). So deferring this doesn't help at all, and this elides async from everything that follows.


return new(document, text, model, root, index);
}
}

public readonly Document Document;
public readonly SourceText Text;
public readonly SemanticModel SemanticModel;
public readonly SyntaxNode Root;
public readonly ISyntaxFactsService SyntaxFacts;
public readonly SyntaxTreeIndex SyntaxTreeIndex;

private readonly ConcurrentDictionary<SyntaxNode, SymbolInfo> _symbolInfoCache = [];
private readonly ConcurrentDictionary<string, ImmutableArray<SyntaxToken>> _identifierCache;

private ImmutableHashSet<string>? _aliasNameSet;
private ImmutableArray<SyntaxToken> _constructorInitializerCache;

private FindReferenceCache(Document document, SemanticModel semanticModel, SyntaxNode root)
private FindReferenceCache(
Document document, SourceText text, SemanticModel semanticModel, SyntaxNode root, SyntaxTreeIndex syntaxTreeIndex)
{
Document = document;
Text = text;
SemanticModel = semanticModel;
Root = root;
SyntaxTreeIndex = syntaxTreeIndex;
SyntaxFacts = document.GetRequiredLanguageService<ISyntaxFactsService>();

_identifierCache = new(comparer: semanticModel.Language switch
{
LanguageNames.VisualBasic => StringComparer.OrdinalIgnoreCase,
Expand All @@ -81,10 +96,8 @@ public SymbolInfo GetSymbolInfo(SyntaxNode node, CancellationToken cancellationT
return null;
}

public async ValueTask<ImmutableArray<SyntaxToken>> FindMatchingIdentifierTokensAsync(
Document document,
string identifier,
CancellationToken cancellationToken)
public ImmutableArray<SyntaxToken> FindMatchingIdentifierTokens(
string identifier, CancellationToken cancellationToken)
{
if (identifier == "")
{
Expand All @@ -99,53 +112,32 @@ public async ValueTask<ImmutableArray<SyntaxToken>> FindMatchingIdentifierTokens
if (_identifierCache.TryGetValue(identifier, out var result))
return result;

// It's very costly to walk an entire tree. So if the tree is simple and doesn't contain
// any unicode escapes in it, then we do simple string matching to find the tokens.
var info = await SyntaxTreeIndex.GetRequiredIndexAsync(document, cancellationToken).ConfigureAwait(false);

// If this document doesn't even contain this identifier (escaped or non-escaped) we don't have to search it at all.
if (!info.ProbablyContainsIdentifier(identifier))
if (!this.SyntaxTreeIndex.ProbablyContainsIdentifier(identifier))
return [];

return await ComputeAndCacheTokensAsync(this, document, identifier, info, cancellationToken).ConfigureAwait(false);
// If the identifier was escaped in the file then we'll have to do a more involved search that actually
// walks the root and checks all identifier tokens.
//
// otherwise, we can use the text of the document to quickly find candidates and test those directly.
return this.SyntaxTreeIndex.ProbablyContainsEscapedIdentifier(identifier)
? _identifierCache.GetOrAdd(identifier, _ => FindMatchingIdentifierTokensFromTree())
: _identifierCache.GetOrAdd(identifier, _ => FindMatchingIdentifierTokensFromText());

static async ValueTask<ImmutableArray<SyntaxToken>> ComputeAndCacheTokensAsync(
FindReferenceCache cache, Document document, string identifier, SyntaxTreeIndex info, CancellationToken cancellationToken)
{
var syntaxFacts = document.GetRequiredLanguageService<ISyntaxFactsService>();
var root = await cache.SemanticModel.SyntaxTree.GetRootAsync(cancellationToken).ConfigureAwait(false);

// If the identifier was escaped in the file then we'll have to do a more involved search that actually
// walks the root and checks all identifier tokens.
//
// otherwise, we can use the text of the document to quickly find candidates and test those directly.
if (info.ProbablyContainsEscapedIdentifier(identifier))
{
return cache._identifierCache.GetOrAdd(
identifier, _ => FindMatchingIdentifierTokensFromTree(syntaxFacts, identifier, root));
}
else
{
var text = await document.GetValueTextAsync(cancellationToken).ConfigureAwait(false);
return cache._identifierCache.GetOrAdd(
identifier, _ => FindMatchingIdentifierTokensFromText(syntaxFacts, identifier, root, text, cancellationToken));
}
}

static bool IsMatch(ISyntaxFactsService syntaxFacts, string identifier, SyntaxToken token)
=> !token.IsMissing && syntaxFacts.IsIdentifier(token) && syntaxFacts.TextMatch(token.ValueText, identifier);
bool IsMatch(string identifier, SyntaxToken token)
=> !token.IsMissing && this.SyntaxFacts.IsIdentifier(token) && this.SyntaxFacts.TextMatch(token.ValueText, identifier);

static ImmutableArray<SyntaxToken> FindMatchingIdentifierTokensFromTree(
ISyntaxFactsService syntaxFacts, string identifier, SyntaxNode root)
ImmutableArray<SyntaxToken> FindMatchingIdentifierTokensFromTree()
{
using var _ = ArrayBuilder<SyntaxToken>.GetInstance(out var result);
using var obj = SharedPools.Default<Stack<SyntaxNodeOrToken>>().GetPooledObject();

var stack = obj.Object;
stack.Push(root);
stack.Push(this.Root);

while (stack.TryPop(out var current))
{
cancellationToken.ThrowIfCancellationRequested();
if (current.IsNode)
{
foreach (var child in current.AsNode()!.ChildNodesAndTokens().Reverse())
Expand All @@ -154,7 +146,7 @@ static ImmutableArray<SyntaxToken> FindMatchingIdentifierTokensFromTree(
else if (current.IsToken)
{
var token = current.AsToken();
if (IsMatch(syntaxFacts, identifier, token))
if (IsMatch(identifier, token))
result.Add(token);

if (token.HasStructuredTrivia)
Expand All @@ -172,19 +164,18 @@ static ImmutableArray<SyntaxToken> FindMatchingIdentifierTokensFromTree(
return result.ToImmutableAndClear();
}

static ImmutableArray<SyntaxToken> FindMatchingIdentifierTokensFromText(
ISyntaxFactsService syntaxFacts, string identifier, SyntaxNode root, SourceText sourceText, CancellationToken cancellationToken)
ImmutableArray<SyntaxToken> FindMatchingIdentifierTokensFromText()
CyrusNajmabadi marked this conversation as resolved.
Show resolved Hide resolved
{
using var _ = ArrayBuilder<SyntaxToken>.GetInstance(out var result);

var index = 0;
while ((index = sourceText.IndexOf(identifier, index, syntaxFacts.IsCaseSensitive)) >= 0)
while ((index = this.Text.IndexOf(identifier, index, this.SyntaxFacts.IsCaseSensitive)) >= 0)
{
cancellationToken.ThrowIfCancellationRequested();

var token = root.FindToken(index, findInsideTrivia: true);
var token = this.Root.FindToken(index, findInsideTrivia: true);
var span = token.Span;
if (span.Start == index && span.Length == identifier.Length && IsMatch(syntaxFacts, identifier, token))
if (span.Start == index && span.Length == identifier.Length && IsMatch(identifier, token))
result.Add(token);

var nextIndex = index + identifier.Length;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -35,12 +35,4 @@ public void OnDefinitionFound(ISymbol symbol)
public void OnReferenceFound(ISymbol symbol, ReferenceLocation location)
{
}

public void OnFindInDocumentStarted(Document document)
{
}

public void OnFindInDocumentCompleted(Document document)
{
}
}
Loading
Loading