Create "crossgen pack" for new crossgen2 compiler

[dagood]

PR dotnet/coreclr#27296 creates Microsoft.NETCore.Crossgen2 transport packages for the new crossgen compiler. These need to be shipped as new packs similar to apphost and runtime packs so the SDK can resolve and download the new crossgen for use in builds.

Microsoft.NETCore.App.Crossgen.<RID> is what I'm thinking for the new pack ID. There's some uncertainty about the name of the tool: dotnet/coreclr#27296 (comment). The Crossgen part may need tweaking.

When the old crossgen is eventually phased out, it will be removed from the runtime pack. crossgen2 won't take crossgen's place in the runtime pack. It isn't considered good that crossgen was in the runtime pack to begin with, so this is an opportunity to fix that.

This new pack should have the same set of installers/packages as runtime packs.

The transport package isn't built for the full set of NETCoreApp RIDs yet, so we'll match that in Core-Setup for now.

/cc @fadimounir @nguerrera @dleeapho @MichaelSimons

[dagood]

Finally working on this starting today, my dotnet/runtime official build infra and build uptake backlog is out of critical issues for now. 😄

/cc @NikolaMilosavljevic

[fadimounir]

cc @dotnet/crossgen-contrib FYI

[dagood]

Quick update. Got sidetracked by issues related to this month's release but starting to iterate on changes now.

Context: The CoreCLR build zips up its artifacts to give to the Installer part of the build, where these packs are made. (Two separate machines in the official build.) This includes the crossgen2 directory with ~230 files (not just the 10 that are packaged up in the current Microsoft.NETCore.Crossgen2). This transfer directory doesn't include the runtimeconfig.

What I'm planning to do to keep things simple is move the runtimeconfig generation code and the 10 file selection code to Installer. The Microsoft.NETCore.Crossgen2 infra can be deleted since it turns out a transport package isn't necessary in the consolidated repo. (We can produce both for a while if it would be disruptive to stop building the transport pack at this point--e.g. if you're using it for dev workflows.)

I can also go ahead and package up all the files in the crossgen2 dir and skip the runtimeconfig. I'm not sure how far out the goal is of having crossgen2 be a self-contained app.

[fadimounir]

I can also go ahead and package up all the files in the crossgen2 dir and skip the runtimeconfig. I'm not sure how far out the goal is of having crossgen2 be a self-contained app.

The long term plan is that we'll eventually ship crossgen2 as a self-contained app package, because this will be especially useful for any cross-platform/cross-architecture compilation.
For .NET 5, there are only 2 simple supported scenarios: linux-x64->linux-x64 and win-x64 to win-x64 publishing, so we decided to minimize the size of the crossgen2 package by making it a framework-dependent application, and host it on the same runtime that comes with the SDK. Minimizing the size of the package is a good thing for customers, so they wouldn't have to download a giant compiler package.

[dagood]

At the moment it looks like crossgen2 is being built on linux-musl-x64 (Alpine) as well, is that a bug?

Made pretty good progress on this yesterday, I have dotnet/arcade#4650 open on the Arcade side to add support once merged, and #1859 to uptake. (An Arcade SDK update will need to be merged before it can clear CI.) It's looking like a few days until this gets produced by an official build.

[fadimounir]

At the moment it looks like crossgen2 is being built on linux-musl-x64 (Alpine) as well, is that a bug?

No. I think the bug is that I probably forgot to create a package for linux-musl-x64 :)

[dagood]

Ah, well no worries then, what I've done so far is already set up to produce it. 😄

[dagood]

To update this thread: the packs will now be "crossgen2 packs", Microsoft.NETCore.App.Crossgen2.<rid>, per dotnet/arcade#4650 (comment).

[dagood]

This was completed by #1859 Jan 17, I missed closing this issue.

A few tracking issues for cleanup:

• Stop producing Crossgen2 transport package (use Crossgen2 pack downstream instead) #1869 "Stop producing Crossgen2 transport package (use Crossgen2 pack downstream instead)"
• Integrate Crossgen2 pack infrastructure into the shared framework tooling in Arcade #1867 "Integrate Crossgen2 pack infrastructure into the shared framework tooling in Arcade"