forked from dotnet/fsharp
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
672e3a2
commit 5ba78e8
Showing
2 changed files
with
55 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,54 @@ | ||
# Parallel Type-Checking in F# | ||
|
||
This document describes the idea and implementation details for parallel type-checking of files in the F# compiler. | ||
|
||
## Context and the current state of the compiler | ||
|
||
Performance of F# compilation and code analysis is not very high. | ||
|
||
Recent improvements not-withstanding, there are still low-hanging fruit that can be introduced. | ||
|
||
One idea for such an improvement was originally described in https://github.com/dotnet/fsharp/discussions/11634 by @kerams . | ||
This is going to be the main topic of this page. | ||
|
||
### Current state of type-checking | ||
|
||
One of the main phases of compilation is type-checking. Depending on the project in question, it can take up to and exceed 50% of the total compilation time. | ||
Currently, by default all files in a project are type-checked in sequence, one-by-one, leading to increased compilation wall-clock time. | ||
|
||
### Recent addition - "Parallel type checking for impl files with backing sig files" | ||
|
||
A recent [change](https://github.com/dotnet/fsharp/pull/13737) introduced in the compiler allows for parallel type-checking of all `.fs` files backed by `.fsi` files. Such `.fs` files by definition cannot be depended upon by any other files w.r.t. type-checking, since all the type-checking information is exposed by the corresponding `.fsi` files. | ||
|
||
The new feature, when enabled, allows partial parallelisation of type-checking as follows: | ||
1. All `.fsi` files and `.fs` files without backing `.fsi` files are type-checked in sequence, as before. | ||
2. Then all `.fs` files with backing `.fsi` files are type-checked in parallel. | ||
|
||
For a project that uses `.fsi` files throughout, such as the `FSharp.Compiler.Service` project, this presents a major speedup. | ||
|
||
Some data points: | ||
- [Fantomas](https://github.com/fsprojects/fantomas) solution: total build time 17.49s -> 14.28s - [link](https://github.com/dotnet/fsharp/pull/13737#issuecomment-1223637818) | ||
- F# codebase build time: 112s -> 92s - [link](https://github.com/dotnet/fsharp/pull/13737#issuecomment-1223386853) | ||
|
||
#### Enabling the feature | ||
|
||
The feature is opt-in and can be enabled in the compiler via a CLI arg & MSBuild property. | ||
|
||
## The importance of using Server GC for parallel work | ||
|
||
By default .NET processes use Workstation GC, which is single-threaded. What this means is it can become a bottleneck for highly-parallel operations, due to increased GC pressure and the cost of GC pauses being multiplied by the number of threads waiting. | ||
That is why when increasing parallelisation of the compiler and the compiler service it is important to note the GC mode being used and consider enabling Server GC. | ||
|
||
Below is an example showing the difference it can make for a parallel workflow. | ||
|
||
### Parallel projects analysis results for a synthetic solution | ||
| GC Mode | Processing Mode | Time | | ||
|-------------|-----------------|-----------------------------------------| | ||
| Workstation | Sequential | 16005ms | | ||
| Workstation | Parallel | 10849ms | | ||
| Server | Sequential | 14594ms (-9% vs Workstation Sequential) | | ||
| Server | Parallel | 2659ms (-75% vs Workstation Parallel) | | ||
|
||
For more details see https://github.com/dotnet/fsharp/pull/13521 | ||
|
||
### |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters