RFC: Transition Cranelift and Wasmtime to new x86-64 backend by default. #10
Conversation
| [open-questions]: #open-questions | ||
|
|
||
| 1. How much compile-time and runtime performance degradation should we | ||
| accept, if any, before moving forward with this transition? |
There was a problem hiding this comment.
In order to help guiding the answer to this question, how about another question: from the performance analysis that we've made on the new backend, it would be interesting to get a rough idea of the speedups we can reach, and with which associated cost? What I mean is, if we think we can get e.g. a 10% speedup easily (famous last words, but imagine in e.g. one or two months of work), then it makes it easier to accept a degradation in the short-term.
|
I spent some time today running Sightglass and looking at the results. The following numbers are from Compilation timeExecution timeThoughtsThis is somewhat surprising to me; earlier measurements showed different results. For example, on Lucet's benchmarks when I last measured: on Dec 1, I recorded a 20% geomean perf improvement on that suite. I've generally measured compile time to improve relative to the old backend, e.g. about 1.5-2x last summer, though I haven't done a measurement recently. So, something seems to have regressed; all the more motivation for continuous benchmarking. I haven't tried to bisect the shift. That said, we still see a modest gain in execution speed on most benchmarks -- geomean of 4.6% better. A compilation time of 18% slower is a little more worrying -- a quick look with I would propose that we still move forward with the transition, but this increases the urgency with which I'm working on this! |
Motion to finalize with a disposition to mergeSo there is not much discussion here, but there also seem to be few objections. Given that runtime perf is generally positive, compatibility check-boxes are complete, and the maintenance story improves greatly once this moves forward, I'm proposing that we merge this RFC. Stakeholders sign-offTagging all employees of BA-affiliated companies who have committed to the Wasmtime or Lucet repos in the last three months, plus anyone who has given feedback on this PR as a stakeholder. FastlyIntelIBMMozillaOther Stakeholders |
|
I think the RFC looks great. Considering no alarms of missing features so far from those who have made the switch, 5 can be done after the switch before the removal of the legacy backend. |
Entering Final Call Period
The FCP will end on Mon Apr 5. |
|
I agree as well. |
|
Thanks all! As we actually have a signoff from every group of stakeholders, it looks like we can close the FCP early and go ahead with the merge:
|
As per bytecodealliance/rfcs#10, the first step in our transition to the new Cranelift x86-64 backend by default in Wasmtime is to switch our fuzzers and monitor for any breakage. This PR moves all Wasmtime fuzz targets over to use the new backend. The differential old-vs-new-backend target still works by dynamically selecting both backends (the build feature just changes the default, which is what all the other fuzz targets use). We should watch for ~a week or so, I think, on oss-fuzz, to make sure this sticks without issue before moving further.
In bytecodealliance/rfcs#10, we have outlined a process by which we're switching to a new compiler backend by default. The first step in this process is to switch our fuzzing targets to use the new backend and wait for any issues. This PR adds the Cargo feature that enables the new backend in all fuzzing targets.
In bytecodealliance/rfcs#10, we have outlined a process by which we're switching to a new compiler backend by default. The first step in this process is to switch our fuzzing targets to use the new backend and wait for any issues. This PR adds the Cargo feature that enables the new backend in all fuzzing targets.
Rendered
This RFC proposes to switch the default Cranelift compiler backend for the
cranelift-codegencrate (i.e., use of Cranelift as a library) and for Wasmtime from the old x86-64 backend to the new x86-64 backend. Details on how we should decide, what steps we should take to build confidence, ensure backwards compatibility, and gradually switch over are proposed.