From 5ba78e86310fe681f0a729f15cbd4dd0abc23f94 Mon Sep 17 00:00:00 2001 From: janusz Date: Tue, 18 Oct 2022 23:45:37 +0100 Subject: [PATCH] Start writing docs --- tests/FSharp.Compiler.Service.Tests2/Docs.md | 54 +++++++++++++++++++ .../FSharp.Compiler.Service.Tests2.fsproj | 1 + 2 files changed, 55 insertions(+) create mode 100644 tests/FSharp.Compiler.Service.Tests2/Docs.md diff --git a/tests/FSharp.Compiler.Service.Tests2/Docs.md b/tests/FSharp.Compiler.Service.Tests2/Docs.md new file mode 100644 index 00000000000..ba70a0b8ed8 --- /dev/null +++ b/tests/FSharp.Compiler.Service.Tests2/Docs.md @@ -0,0 +1,54 @@ +# Parallel Type-Checking in F# + +This document describes the idea and implementation details for parallel type-checking of files in the F# compiler. + +## Context and the current state of the compiler + +Performance of F# compilation and code analysis is not very high. + +Recent improvements not-withstanding, there are still low-hanging fruit that can be introduced. + +One idea for such an improvement was originally described in https://github.com/dotnet/fsharp/discussions/11634 by @kerams . +This is going to be the main topic of this page. + +### Current state of type-checking + +One of the main phases of compilation is type-checking. Depending on the project in question, it can take up to and exceed 50% of the total compilation time. +Currently, by default all files in a project are type-checked in sequence, one-by-one, leading to increased compilation wall-clock time. + +### Recent addition - "Parallel type checking for impl files with backing sig files" + +A recent [change](https://github.com/dotnet/fsharp/pull/13737) introduced in the compiler allows for parallel type-checking of all `.fs` files backed by `.fsi` files. Such `.fs` files by definition cannot be depended upon by any other files w.r.t. type-checking, since all the type-checking information is exposed by the corresponding `.fsi` files. + +The new feature, when enabled, allows partial parallelisation of type-checking as follows: +1. All `.fsi` files and `.fs` files without backing `.fsi` files are type-checked in sequence, as before. +2. Then all `.fs` files with backing `.fsi` files are type-checked in parallel. + +For a project that uses `.fsi` files throughout, such as the `FSharp.Compiler.Service` project, this presents a major speedup. + +Some data points: +- [Fantomas](https://github.com/fsprojects/fantomas) solution: total build time 17.49s -> 14.28s - [link](https://github.com/dotnet/fsharp/pull/13737#issuecomment-1223637818) +- F# codebase build time: 112s -> 92s - [link](https://github.com/dotnet/fsharp/pull/13737#issuecomment-1223386853) + +#### Enabling the feature + +The feature is opt-in and can be enabled in the compiler via a CLI arg & MSBuild property. + +## The importance of using Server GC for parallel work + +By default .NET processes use Workstation GC, which is single-threaded. What this means is it can become a bottleneck for highly-parallel operations, due to increased GC pressure and the cost of GC pauses being multiplied by the number of threads waiting. +That is why when increasing parallelisation of the compiler and the compiler service it is important to note the GC mode being used and consider enabling Server GC. + +Below is an example showing the difference it can make for a parallel workflow. + +### Parallel projects analysis results for a synthetic solution +| GC Mode | Processing Mode | Time | +|-------------|-----------------|-----------------------------------------| +| Workstation | Sequential | 16005ms | +| Workstation | Parallel | 10849ms | +| Server | Sequential | 14594ms (-9% vs Workstation Sequential) | +| Server | Parallel | 2659ms (-75% vs Workstation Parallel) | + +For more details see https://github.com/dotnet/fsharp/pull/13521 + +### \ No newline at end of file diff --git a/tests/FSharp.Compiler.Service.Tests2/FSharp.Compiler.Service.Tests2.fsproj b/tests/FSharp.Compiler.Service.Tests2/FSharp.Compiler.Service.Tests2.fsproj index 06bc0a2402d..6672b1f894e 100644 --- a/tests/FSharp.Compiler.Service.Tests2/FSharp.Compiler.Service.Tests2.fsproj +++ b/tests/FSharp.Compiler.Service.Tests2/FSharp.Compiler.Service.Tests2.fsproj @@ -35,6 +35,7 @@ PreserveNewest +