RFC 42: QUIC server in WPT

[Hexcles]

Rendered

This RFC addresses web-platform-tests/wpt#19114

cc @yutakahirano

[jgraham]

@annevk Assuming I look at the implmentation details, is there someone from Mozilla who should give feedback on whether this meets any requirements from our side?

[jgraham]

This isn't a filename flag because it isn't about loading the tests themselves over quic, but about being able to filter out quic-requiring tests where the server isn't available, correct? So we might hope to remove this in the future. Is there some way to enforce this; it seems like it's easy for a test not to add the meta element and still work correctly in configurations where quic is enabled?

[Hexcles]

This isn't a filename flag because it isn't about loading the tests themselves over quic, but about being able to filter out quic-requiring tests where the server isn't available, correct?

Correct. I thought about using .quic.html, similar to .h2, but realized that they are in fact different. The test themselves should not be loaded via QUIC.

So we might hope to remove this in the future.

Probably, when all major vendors have WebTransport implementations and we run these tests by default.

Is there some way to enforce this; it seems like it's easy for a test not to add the meta element and still work correctly in configurations where quic is enabled?

Great point! I don't have good ideas right now. Any thoughts?

[jgraham]

Are we happy with the mainteinance story of this library? How did you decide that it has the features we need to give the low-level protocol access that's required for writing tests?

[Hexcles]

@yutakahirano knows more details here.

[jgraham]

So it seems like updating the version here is going to be an infra-breaking change for vendors because work will be required on the vendor side to update to the new version (e.g. updating the deps file or uploading the wheel to the internal pypi mirror or whatever each vendor does). That seems like a pretty big concern to me. If we did vendor the source, we could have logic like "check if the correct version is available, or pip install, or build from source". Maybe there are other possibilities to address this?

[Hexcles]

I'm thinking about this from two perspectives:

• For average human users, it's most likely they are using a platform supported by PyPI wheels, and the dependencies will be installed into _venv using pip automatically. Vendoring the source isn't very useful for them.
• For browser infras, AFAIK both Firefox and Chromium discourage or even disallow building a native Python extension at run time; these dependencies will have to come from a PyPI-like repo. If that's the case, vendoring the source doesn't seem helpful, either.

I agree that this will incur a maintenance burden on the vendors if they want to run QUIC tests. (I'll acknowledge this in the risks section.) Meanwhile, I don't see vendoring the source code would help the browser vendors if my understanding above is correct -- someone would still need to build and distribute the new version.

[jgraham]

I don't think our test environment forbids building a native extension, although it also seems likely that the required libraries aren't available. What would definitely be possible (although not tivial) is adding the build step to the Firefox build and distributing the resulting package to the test machines; we already have various test-only binaries that need to be produced.

What worries me about the proposed setup is that updating the library version is a sync-blocking change for Gecko since we're a) using the tools/ directory directly and b) don't allow PyPI access from test machines for robustness reasons.

[Hexcles]

That's true for Chromium, too. I imagine that in the future when we need to update these native dependencies, we will need to create a PR that updates the requirements.txt for venv, and ask each vendor to prepare the packages in their infras, before merging the PR. Native dependencies are difficult to deal with in general; I'm intentionally making some tradeoff here between ongoing maintenance costs and complexity.

Do you have a more concrete alternative proposal? If you think vendoring the source into the codebase is easier for Firefox infra, then I'd be happy to add that as a fallback to wheels, too. I just didn't want to vendor them if it wouldn't be used by anyone.

[jgraham]

I think that adding the source would in principle be better for Mozilla, since we could in theory add it to the build system. In practice I don't know that we'd actually get around to doing that :/ It probably depends on how hard it is vs doing manual updates. But I think I'd like us to do it anyway.

In any case I think the process probably demands an RFC for version updates, since it's a change that affects multiple vendors. That's pretty heavyweight, but this really is the kind of change that can break integrations.

[martinthomson]

I think that this is a reasonable plan, but I can't comment on the specifics until I have more information about webtransport. In that respect, this is probably well premature.

If you regard the current design being proposed by Google as a fait accompli, this is probably enough, but the extra effort required to build something usable on top of a generic QUIC stack could turn out to be such that this approach becomes unwieldy.

[Hexcles]

Thanks for the constructive and insightful feedback!

I want to highlight one point: this has NOT been implemented yet, and the design of the QUIC server itself (internal architecture, handler API, etc.) is intentionally vague because I don't have the details yet and we will probably iterate on the design once we start prototyping -- @yutakahirano and the team will work on this.

The key part of this RFC is the integration between the QUIC server and the rest of WPT infra, which should not need to change regardless of the detailed design of the QUIC server itself. This RFC paves the way for prototyping and eventually productionizing the QUIC server within WPT with minimum overhead. We can send out another RFC when we are happy with a detailed server design or implementation.

[Hexcles]

@jgraham I've updated the RFC:

1. Switch to off-by-default
2. Clarify the scope of the RFC (i.e. not including server APIs)
3. Clarify dependency management and list it as a "risk"

Could you take another look? There are a few open questions now:

• Deferring the detailed server API design
• Unable to strictly enforce that a test without the QUIC tag does not need QUIC
• Dependency management

Do you see any of these as blocking? These are some genuinely good questions to which I don't have perfect answers to. The currently proposed design makes some trade-offs in these areas.

[Hexcles]

ping @jgraham

[jgraham]

In response to

• Deferring the detailed server API design

I think this is OK as long as it's off by default. By the time we want broad contributions there needs to be a clear, documented, API for people to write tests against. So that would be before anyone ships QUIC-dependent features by default, for example. I think a commitment to make a server API RFC, and an expected timeline for that in terms of external factors (e.g. implementation status) should be part of this RFC.

• Unable to strictly enforce that a test without the QUIC tag does not need QUIC

This worries me. We should at least lint for .quic.py in files that aren't marked as requiring quic. But I don't think that there's any provision at the moment for examining dependent files, so a test requiring helper.html that requests helper.quic.py would be harder to catch. I think at the minimum a commitment to add some linting should be part of this RFC.

One other possibility that just occurred to me is to require everything that accesses quic to be in a quic directory and have the server itself reject requests without quic/ in the path. Then you could lint for any occurrence of the string quic outside a quic directory, which would do something to prevent test/foo.html requesting test/quic/helper.html. That's more pain in the long run, but could be relaxed when we turn this on by default.

[jgraham]

I think that adding the source would in principle be better for Mozilla, since we could in theory add it to the build system. In practice I don't know that we'd actually get around to doing that :/ It probably depends on how hard it is vs doing manual updates. But I think I'd like us to do it anyway.

In any case I think the process probably demands an RFC for version updates, since it's a change that affects multiple vendors. That's pretty heavyweight, but this really is the kind of change that can break integrations.

[Hexcles]

@jgraham I think I've addressed all the remaining comments. Please take another look at the latest commit. Thanks!

[Hexcles]

ping @jgraham

[jgraham]

Just a minor fix. I think this is OK now, but I foresee this generating future work to improve the checks for metadata accuracy, and to allow upgrades.

[jgraham]

I think this is OK to land as the basis for experimental support, with the expectation that there will be future RFCs to nail down the API surface and at the point at which we want to switch to on-by-default.

[Hexcles]

Heads up: if there's no more comment I'll be merging this in 3 days (which will be a full week after the last comment).