creating-a-source-generator-part-1-creating-an-incremental-source-generator/

[giscus[bot]]

creating-a-source-generator-part-1-creating-an-incremental-source-generator/

In this post I walk through how to create a practical .NET 6 incremental source generator: an enum extensions class with a fast ToString() implementation

https://andrewlock.net/creating-a-source-generator-part-1-creating-an-incremental-source-generator/

[CyrusNajmabadi]

Hi there. One of the incremental source generator api authors here. If possible, if recommend adding a note (or followup post) mentioning that your can eliminate several steps (and have them run much more efficiently) if you use this new api: https://github.com/dotnet/roslyn/blob/7d64f6edcc0ee5fe4399a7966e1a6b2bd3ef9005/src/Compilers/Core/Portable/SourceGeneration/Nodes/SyntaxValueProvider_ForAttributeWithMetadataName.cs#L78

Thanks very much!

[andrewlock]

Hi Cyrus, thanks, it's been on my radar to write a post about this for a while! I'll add that note shortly.

Looking back at this now, of more concern is how many bad behaviour's it's doing - returning Syntax nodes, including the Compilation in the pipeline, collecting all the results into one 😅 I'll definitely fix these too

[Miljankg]

Hi Andrew,

Thanks a lot for this tutorial. It helped me a lot. I do have one question regarding the pipeline you built. In the part where Syntax Provider is created (context.SyntaxProvider.CreateSyntaxProvider) there are two parameters passed, predicate and transform. You stated that the predicate should be efficient, and opted for brief and quick filtering by the number of attributes of a particular syntax node. Then in the transform, nodes are filtered further by looking for the particular attribute.

How much of a problem would it be if I would move the complete filtering in your example, into the predicate and leave the transform to do only that, getting and transforming the data to what is needed further down the pipeline? Would you consider such a predicate inefficient? Talking strictly about your example.

Thanks in advance!
Miljan

[Miljankg]

Just to provide some info on this in case it helps someone. The predicate serves for examining Syntax Nodes in isolation. The name of the attribute is available here (e.g. SomeAttribute), but the namespace cannot be established from Syntax Node alone. Therefore, the predicate has only a simple filtering of Syntax Nodes.

For full attribute name (with namespace), Semantic Model is needed, where Symbols associated with Syntax Node can be examined. This is available in the transform method.

For more information on these visit:

• Incremental Generators in GitHub
• This blogpost I found useful
• Andew's explanation of generator pipeline

[goldytech]

I’m unable to get this working on latest Rider version on MacOS. For me the source generator was able to generate code after I decorated my class with the marker attribute, even intellisense also worked. But when I try to run or compile the console app which consumes the source generator via project reference, it gives me error of “Cannot resolve the method which is auto generated

[drdamour]

in section 3, what's the reason to define the enum as a source generated type instead of just a type in the assembly that can be referenced in downstream project? i feel like i'm losing simple compiler/type checking stuff i'm familiar with by defining the attribute in a string, but i'm guessing there's an advantage to doing it that way.

[drdamour]

derp u answered this in https://andrewlock.net/creating-a-source-generator-part-7-solving-the-source-generator-marker-attribute-problem-part1/ & https://andrewlock.net/creating-a-source-generator-part-8-solving-the-source-generator-marker-attribute-problem-part2/

[andyolivares]

Hi Andrew:

First of all, thank you for these articles. They helped me to design my own source generator for a couple of parts of my system that heavily relied on Reflection. I've seen improvements of up to ~90x in speed at some places.

The "simple" generator (a single generator lib project) worked just fine, but since I splitted my generator into a few libraries to share code between 3 different generators, things went south and I've been unable to get it working again.

My project configuration is almost identical to yours, but VS2022 fails to generate any code nor compile (my generated classes are not being found).

I've been reading online and tried a few things. Most notably, I've been through some of the things mentioned here:
https://turnerj.com/blog/the-pain-points-of-csharp-source-generators

Added what they mention in Pain Point 3 and now my generator seems to create my classes (at least VS2022 compiles my project) but can't see any generated code in the IDE (as with the "simple" generator).

I'm not using the generator as a Nuget package but as a project reference for now. As I said before, the "simple" case being just one project, worked just fine, but when I separated the generator into different projects and started adding references between them, it stopped working.

Did you experience any of this? Do you know where to look at?

Thank you in advance!

Andy

[andrewlock]

Hey, short answer, I think you're going to be fighting with this approach - it's not something I've tried and honestly, I'm not going to try as I won't be surprised if it just doesn't work 😬

If you don't need type unification, one thing you could try doing is using a shared project (similar to this), or adding the files as links. It doesn't really solve your issue, but you may be able skirt around it!

[andyolivares]

Hi Andrew, thank you for your reply. At the end, I've managed to solve the issue and now my generators are doing their job pretty well. For anyone's reference I had to do something quite "hacky" (really don't know if that's the right approach, but it works):

<PropertyGroup>
  <TargetFramework>netstandard2.0</TargetFramework>
  <LangVersion>9.0</LangVersion>
  <IncludeBuildOutput>false</IncludeBuildOutput>
  <IsRoslynComponent>true</IsRoslynComponent>
</PropertyGroup>

<ItemGroup>
  <PackageReference Include="Microsoft.CodeAnalysis.Analyzers" Version="3.3.3" PrivateAssets="all" />
  <PackageReference Include="Microsoft.CodeAnalysis.CSharp" Version="4.3.0" PrivateAssets="all" />
  <ProjectReference Include="..\Generators.Common\Generators.Common.csproj" PrivateAssets="all" ExcludeAssets="runtime" />
</ItemGroup>

<PropertyGroup>
  <GetTargetPathDependsOn>$(GetTargetPathDependsOn);GetDependencyTargetPaths</GetTargetPathDependsOn>
</PropertyGroup>

<Target Name="GetDependencyTargetPaths">
  <ItemGroup>
    <TargetPathWithTargetPlatformMoniker Include="$(SolutionDir)src\Generators.Common\bin\$(Configuration)\netstandard2.0\Generators.Common.dll" IncludeRuntimeDependency="false" />
  </ItemGroup>
</Target>

As you can see, I had to include my shared project's DLL with TargetPathWithTargetPlatformMoniker. Not sure what it really does under the hood, but it did the trick.

Now, I'm having troubles trying to generate any code from dotnet build (as in VS2022 it works just fine).

Regards,

Andy

[disqus-bot]

Jason Bock commented on Disqus at December 09 2021, 05:59

---

In the first method in the pipeline, could you search for AttributeSyntax nodes that have the name "EnumExtensions" or "EnumExtensionsAttribute"? Then you can find the enum it's associated with later. But then you know in your first pass that you have what you're looking for.

Great article BTW :)

[disqus-bot]

David Garland commented on Disqus at January 29 2022, 08:54, in reply to Jason Bock's comment

---

I briefly came to the same conclusion, and indeed it seems another more recent blog post does it the way you describe: https://www.thinktecture.com/en/net/roslyn-source-generators-introduction/

However, the author of this article appears to emphasize speed of execution (..though they ignore the cancellation token, a fault shared by the other article as well). If we think about it from that angle, I think the method the author of this article presents is more likely to be performant, under the following observations:
- String comparison is linear time in the worst case, and that appears to be the only way to compare identifiers. I would think having pre-hashed identifiers would be good for metaprogramming like this, but I don't see anything in the sparse API documentation that suggests that they do this. If there *is* a better way of doing it, I'd like to be pointed to it so I can start using it myself. (Maybe SemanticModel offers this somehow? It seems that you get a new one at each invocation of the transform lambda, is the issue I see with that idea..)
- The number of classes that have attributes in your program is generally less than the number of attributes in your program, as they can be applied to other things as well, like methods.
So if we assume string comparisons are expensive, the author's technique should be doing fewer of them and be strictly faster. Maybe in the real world most string comparisons are O(1) just based off of length mismatches and it doesn't matter much, but at the very least maybe this reveals their thinking. Your method is definitely simpler in terms of the amount of code necessary, though.

[disqus-bot]

David Garland commented on Disqus at January 29 2022, 09:00, in reply to David Garland's comment

---

I'd also like to note that the other article does a better job of extracting the string from the identifier rather than using ToDisplayString() like this one does, though that article also does it as a loop unnecessarily when the ".Right" method of a QualifiedNameSyntax is guaranteed to return a SimpleNameSyntax. So it could have just been this, if I'm not misunderstanding (forgive the formatting, Disqus seems to be ignoring my indentation):

private static string? ExtractName(NameSyntax? name) =>
name switch
{
IdentifierNameSyntax ins => ins.Identifier.Text,
QualifiedNameSyntax qns => qns.Right.Identifier.Text,
_ => null
};

So really both articles have their ups and downs.

[disqus-bot]

Rich Armstrong commented on Disqus at June 14 2022, 04:12

---

From step 6, Implementing the pipeline stages:

private const string EnumExtensionsAttribute = "...";

I'm guessing there were once plans for this little const, but they never got realized?

[andrewlock]

Sock (@andrewlock) commented on Disqus at June 28 2022, 09:56, in reply to Rich Armstrong's comment

---

oops yeah, removed it now, thanks

[disqus-bot]

Raymond Osterbrink commented on Disqus at June 22 2022, 04:40

---

This tutorial is brilliant and was helping me so much in understanding the concept of the IIncrementalGenerator but still, there is one major question I could not answer: is it possible to access multiple projects inside my solution by the Generator, e.g. getting triggerd by changes in a shared Library and creating different code in the UI-project and the Backend-Project based on the changes?
I did so in the ISourceGenerator for .Net5 but only with a rather hacky workaround, by referencing the ISourceGenerator only in the UI/Backend-Project and accessing the shared Library via Compilation.SourceModule.ReferencedAssemblySymbols; but I would prefere not to do so, with the IIncrementalGenerator because of performance issues, also I have no Idea how to trigger the changes from Library, when referencing the Generator not inside the shared Library, which happens to get manipulated.

[luizfbicalho]

Nice post , I always read it again to improve my generators

I will change my generators to use global namespaces and I got a point here

what is the best practice to generics and nullable?

global::System.Threading.Tasks.Task<global::System.String?> x = null;
global::System.Threading.Tasks.Task<string?> x2 = null;
global::System.Threading.Tasks.Task<global::System.Nullable<global::System.String>> x3 = null;

[CyrusNajmabadi]

use string?

[luizfbicalho]

Is there any extension or nuget package that format any type as global to use in source generators @andrewlock?