[wasm][aot] Run the high resource AOT tests on build machine itself (#…

…67017) Some library tests fail to AOT on helix by getting oomkill'ed. Since the build machine have more resources than helix, we can AOT these specific library tests there, and send the built output to helix to run. Specifically: - `System.Text.Json.SourceGeneration.Roslyn3.11.Tests` - `System.Text.Json.SourceGeneration.Roslyn4.0.Tests` - `System.Text.Json.Tests` - `Microsoft.Extensions.Logging.Generators.Roslyn3.11.Tests` This adds two new jobs to `runtime-wasm`, and for rolling builds - `LibraryTests_HighResource_AOT` for linux, and windows. Fixes #65356 Fixes #65411 Fixes #61524 Fixes #66647
dotnet · Mar 25, 2022 · 286492c · 286492c
1 parent cc4d26d
commit 286492c
Show file tree

Hide file tree

Showing 6 changed files with 73 additions and 18 deletions.
diff --git a/eng/pipelines/common/global-build-job.yml b/eng/pipelines/common/global-build-job.yml
@@ -113,7 +113,6 @@ jobs:
         ${{ if ne(parameters.buildingOnSourceBuildImage, true) }}:
           value: ''
 
-
       - ${{ each variable in parameters.variables }}:
         - ${{ variable }}
 
@@ -192,6 +191,9 @@ jobs:
       inputs:
         PathtoPublish: '$(Build.SourcesDirectory)/artifacts/log/'
         PublishLocation: Container
-        ArtifactName: Logs_Build_${{ parameters.osGroup }}_${{ parameters.osSubGroup }}_${{ parameters.archType }}_${{ parameters.buildConfig }}_${{ parameters.nameSuffix }}
+        ${{ if ne(parameters.osGroup, 'Browser') }}:
+          ArtifactName: Logs_Build_${{ parameters.osGroup }}_${{ parameters.osSubGroup }}_${{ parameters.archType }}_${{ parameters.buildConfig }}_${{ parameters.nameSuffix }}
+        ${{ if eq(parameters.osGroup, 'Browser') }}:
+          ArtifactName: Logs_Build_Browser_wasm_${{ parameters.hostedOs }}_${{ parameters.buildConfig }}_${{ parameters.nameSuffix }}
       continueOnError: true
       condition: always()
diff --git a/eng/pipelines/common/templates/wasm-library-aot-tests.yml b/eng/pipelines/common/templates/wasm-library-aot-tests.yml
@@ -3,6 +3,7 @@ parameters:
   extraBuildArgs: ''
   extraHelixArgs: ''
   isExtraPlatformsBuild: false
+  buildAOTOnHelix: true
   nameSuffix: ''
   platforms: []
   runAOT: false 
@@ -18,7 +19,7 @@ jobs:
     platforms: ${{ parameters.platforms }}
     nameSuffix: ${{ parameters.nameSuffix }}
     isExtraPlatforms: ${{ parameters.isExtraPlatformsBuild }}
-    extraBuildArgs: /p:EnableAggressiveTrimming=true /p:BuildAOTTestsOnHelix=true /p:RunAOTCompilation=${{ parameters.runAOT }} ${{ parameters.extraBuildArgs }}
+    extraBuildArgs: /p:EnableAggressiveTrimming=true /p:BuildAOTTestsOnHelix=${{ parameters.buildAOTOnHelix }} /p:RunAOTCompilation=${{ parameters.runAOT }} ${{ parameters.extraBuildArgs }}
     extraHelixArgs: /p:NeedsToBuildWasmAppsOnHelix=true ${{ parameters.extraHelixArgs }}
     alwaysRun: ${{ parameters.alwaysRun }}
     runSmokeOnlyArg: $(_runSmokeTestsOnlyArg)
diff --git a/eng/pipelines/runtime-extra-platforms-wasm.yml b/eng/pipelines/runtime-extra-platforms-wasm.yml
@@ -34,6 +34,18 @@ jobs:
       runAOT: true
       alwaysRun: true
 
+  # High resource AOT Library tests
+  - template: /eng/pipelines/common/templates/wasm-library-aot-tests.yml
+    parameters:
+      platforms:
+        - Browser_wasm
+        - Browser_wasm_win
+      nameSuffix: _HighResource_AOT
+      extraBuildArgs: /p:TestAssemblies=false /p:RunHighAOTResourceRequiringTestsOnly=true
+      buildAOTOnHelix: false
+      runAOT: true
+      alwaysRun: true
+
 #
 # ********** For !rolling builds, IOW - PR builds *************
 # - run everything, if relevant paths changed
@@ -72,6 +84,18 @@ jobs:
       runAOT: true
       alwaysRun: ${{ parameters.isWasmOnlyBuild }}
 
+  # High resource AOT Library tests
+  - template: /eng/pipelines/common/templates/wasm-library-aot-tests.yml
+    parameters:
+      platforms:
+        - Browser_wasm
+        - Browser_wasm_win
+      nameSuffix: _HighResource_AOT
+      extraBuildArgs: /p:TestAssemblies=false /p:RunHighAOTResourceRequiringTestsOnly=true
+      buildAOTOnHelix: false
+      runAOT: true
+      alwaysRun: ${{ parameters.isWasmOnlyBuild }}
+
   # Wasm.Build.Tests
   - template: /eng/pipelines/common/templates/wasm-build-tests.yml
     parameters:

diff --git a/eng/testing/tests.wasm.targets b/eng/testing/tests.wasm.targets
@@ -5,6 +5,10 @@
     <WasmGenerateAppBundle Condition="'$(WasmGenerateAppBundle)' == ''">true</WasmGenerateAppBundle>
     <BundleTestAppTargets>$(BundleTestAppTargets);BundleTestWasmApp</BundleTestAppTargets>
     <DebuggerSupport Condition="'$(DebuggerSupport)' == '' and '$(Configuration)' == 'Debug'">true</DebuggerSupport>
+
+    <!-- set this when provisioning emsdk on CI -->
+    <EMSDK_PATH Condition="'$(EMSDK_PATH)' == '' and '$(ContinuousIntegrationBuild)' == 'true' and '$(MonoProjectRoot)' != ''">$([MSBuild]::NormalizeDirectory($(MonoProjectRoot), 'wasm', 'emsdk'))</EMSDK_PATH>
+
     <!-- Some tests expect to load satellite assemblies by path, eg. System.Runtime.Loader.Tests,
          so, just setting it true by default -->
     <IncludeSatelliteAssembliesInVFS Condition="'$(IncludeSatelliteAssembliesInVFS)' == ''">true</IncludeSatelliteAssembliesInVFS>

diff --git a/src/libraries/tests.proj b/src/libraries/tests.proj
@@ -18,8 +18,28 @@
     <TestAssemblies Condition="'$(TestAssemblies)' == ''">true</TestAssemblies>
     <TestPackages Condition="'$(TestPackages)' == ''">false</TestPackages>
     <TestTrimming Condition="'$(TestTrimming)' == ''">false</TestTrimming>
+
+    <RunHighAOTResourceRequiringTestsOnly Condition="'$(RunHighAOTResourceRequiringTestsOnly)' == ''">false</RunHighAOTResourceRequiringTestsOnly>
+
+    <!-- Don't build samples, and functional tests on EAT, AOT, WBT, and Debugger lanes -->
+    <RunWasmSamples Condition="'$(RunSmokeTestsOnly)' != 'true' and '$(RunHighAOTResourceRequiringTestsOnly)' != 'true' and '$(TargetOS)' == 'Browser' and '$(BuildAOTTestsOnHelix)' != 'true' and  '$(TestWasmBuildTests)' != 'true' and '$(TestWasmDebuggerTests)' != 'true'">true</RunWasmSamples>
   </PropertyGroup>
 
+  <ItemGroup Condition="'$(TargetOS)' == 'Browser'">
+    <!-- https://github.com/dotnet/runtime/issues/65356 - OOM while linking -->
+    <HighAOTResourceRequiringProject Include="$(MSBuildThisFileDirectory)System.Text.Json\tests\System.Text.Json.SourceGeneration.Tests\System.Text.Json.SourceGeneration.Roslyn3.11.Tests.csproj" />
+
+    <!-- https://github.com/dotnet/runtime/issues/65411 - possible OOM when compiling
+         System.Text.Json.SourceGeneration.Roslyn4.0.Tests.dll.bc -> .o -->
+    <HighAOTResourceRequiringProject Include="$(MSBuildThisFileDirectory)System.Text.Json\tests\System.Text.Json.SourceGeneration.Tests\System.Text.Json.SourceGeneration.Roslyn4.0.Tests.csproj" />
+
+    <!-- https://github.com/dotnet/runtime/issues/61524 - OOM while linking -->
+    <HighAOTResourceRequiringProject Include="$(MSBuildThisFileDirectory)System.Text.Json\tests\System.Text.Json.Tests\System.Text.Json.Tests.csproj" />
+
+    <!-- https://github.com/dotnet/runtime/issues/66647 -->
+    <HighAOTResourceRequiringProject Include="$(MSBuildThisFileDirectory)Microsoft.Extensions.Logging.Abstractions\tests\Microsoft.Extensions.Logging.Generators.Tests\Microsoft.Extensions.Logging.Generators.Roslyn3.11.Tests.csproj" />
+  </ItemGroup>
+
   <!-- Samples which are too complex for CI -->
   <ItemGroup Condition="'$(TargetOS)' == 'Browser'">
     <ProjectExclusions Include="$(MonoProjectRoot)sample\wasm\console-node-ts\Wasm.Console.Node.TS.Sample.csproj" />
@@ -37,15 +57,6 @@
 
     <!-- https://github.com/dotnet/runtime/issues/61756 -->
     <ProjectExclusions Include="$(MSBuildThisFileDirectory)System.Text.RegularExpressions\tests\FunctionalTests\System.Text.RegularExpressions.Tests.csproj" />
-
-    <!-- https://github.com/dotnet/runtime/issues/65356 - OOM while linking -->
-    <ProjectExclusions Include="$(MSBuildThisFileDirectory)System.Text.Json\tests\System.Text.Json.SourceGeneration.Tests\System.Text.Json.SourceGeneration.Roslyn3.11.Tests.csproj" />
-    <!-- https://github.com/dotnet/runtime/issues/65411 - possible OOM when compiling
-         System.Text.Json.SourceGeneration.Roslyn4.0.Tests.dll.bc -> .o -->
-    <ProjectExclusions Include="$(MSBuildThisFileDirectory)System.Text.Json\tests\System.Text.Json.SourceGeneration.Tests\System.Text.Json.SourceGeneration.Roslyn4.0.Tests.csproj" />
-
-    <!-- https://github.com/dotnet/runtime/issues/61524 - OOM while linking -->
-    <ProjectExclusions Include="$(MSBuildThisFileDirectory)System.Text.Json\tests\System.Text.Json.Tests\System.Text.Json.Tests.csproj" />
   </ItemGroup>
 
   <!-- Projects that don't support code coverage measurement. -->
@@ -365,11 +376,20 @@
     <ProjectExclusions Remove="$(MSBuildThisFileDirectory)System.Collections\tests\System.Collections.Tests.csproj" />
   </ItemGroup>
 
-  <ItemGroup Condition="'$(RunSmokeTestsOnly)' == 'true'">
-    <ProjectReference Include="$(MSBuildThisFileDirectory)System.Runtime\tests\System.Runtime.Tests.csproj" />
+  <ItemGroup>
+    <ProjectExclusions Condition="'$(RunHighAOTResourceRequiringTestsOnly)' != 'true'"
+                       Include="@(HighAOTResourceRequiringProject)" />
+  </ItemGroup>
+
+  <ItemGroup>
+    <ProjectReference Condition="'$(RunSmokeTestsOnly)' == 'true'"
+                      Include="$(MSBuildThisFileDirectory)System.Runtime\tests\System.Runtime.Tests.csproj" />
+    <ProjectReference Condition="'$(RunHighAOTResourceRequiringTestsOnly)' == 'true'"
+                      Include="@(HighAOTResourceRequiringProject)"
+                      BuildInParallel="false" />
   </ItemGroup>
 
-  <ItemGroup Condition="'$(RunSmokeTestsOnly)' != 'true'">
+  <ItemGroup Condition="'$(RunSmokeTestsOnly)' != 'true' and '$(RunHighAOTResourceRequiringTestsOnly)' != 'true'">
     <ProjectReference Include="$(MSBuildThisFileDirectory)*\tests\**\*.Tests.csproj"
                       Exclude="@(ProjectExclusions)"
                       Condition="'$(TestAssemblies)' == 'true'" />
@@ -448,8 +468,7 @@
     <Samples_BuildInParallel Condition="'$(OS)' != 'Windows_NT'">true</Samples_BuildInParallel>
   </PropertyGroup>
 
-  <!-- Don't build samples, and functional tests on EAT, AOT, WBT, and Debugger lanes -->
-  <ItemGroup Condition="'$(ArchiveTests)' == 'true' and '$(RunSmokeTestsOnly)' != 'true' and '$(TargetOS)' == 'Browser' and '$(BuildAOTTestsOnHelix)' != 'true' and '$(TestWasmBuildTests)' != 'true' and '$(TestWasmDebuggerTests)' != 'true'">
+  <ItemGroup Condition="'$(ArchiveTests)' == 'true' and '$(RunWasmSamples)' == 'true'">
     <ProjectReference Include="$(MonoProjectRoot)sample\wasm\**\*.Sample.csproj"
                       Exclude="@(ProjectExclusions)"
                       BuildInParallel="$(Samples_BuildInParallel)" />

diff --git a/src/mono/wasm/README.md b/src/mono/wasm/README.md
@@ -215,11 +215,13 @@ Bumping Emscripten version involves these steps:
 | libtests          | linux: all,   only-pc | windows: all,   only-pc | linux+windows: all, only-pc          | linux+windows: all, always |
 | libtests eat      | linux: smoke, only-pc | -                       | linux:         all, only-pc          | linux:         all, always |
 | libtests aot      | linux: smoke, only-pc | windows: smoke, only-pc | linux+windows: all, only-pc          | linux+windows: all, always |
-|                   |                       |                         |                                      |                            |
+| high resource aot | none                  | none                    | linux+windows: all, only-pc          | linux+windows: all, always |
 | Wasm.Build.Tests  | linux:        only-pc | windows:        only-pc | linux+windows: only-pc               | linux+windows              |
 | Debugger tests    | -                     | linux+windows:  only-pc | linux+windows: only-pc               | linux+windows              |
 | Runtime tests     | linux:        only-pc | -                       | linux: only-pc                       | linux                      |
 
+* `high resource aot` runs a few specific library tests with AOT, that require more memory to AOT.
+
 ## Rolling build (twice a day):
 
 * `runtime`, and `runtime-staging`, run all the wasm jobs unconditionally, but `EAT`, and `AOT` still run only smoke tests.
@@ -232,7 +234,10 @@ Bumping Emscripten version involves these steps:
 | libtests          | linux: all(v8/chr)        | windows: all          | none                                 | N/A                        |
 | libtests eat      | linux: smoke              | -                     | linux: all                           |                            |
 | libtests aot      | linux: smoke              | windows: smoke        | linux+windows: all                   |                            |
+| high resource aot | none                      | none                  | linux+windows: all                   |                            |
 |                   |                           |                       |                                      |                            |
 | Wasm.Build.Tests  | linux: always             | windows: always       | none                                 |                            |
 | Debugger tests    | -                         | linux+windows: always | none                                 |                            |
 | Runtime tests     | linux: always             | -                     | none                                 |                            |
+
+* `high resource aot` runs a few specific library tests with AOT, that require more memory to AOT.