Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RFC 42: QUIC server in WPT #42

Merged
merged 5 commits into from
Mar 19, 2020
Merged

RFC 42: QUIC server in WPT #42

Changes from 4 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
9057334
Add quic.md
Hexcles Feb 5, 2020
07b7196
Fix typos
Hexcles Feb 5, 2020
3c0b0f1
Address some comments
Hexcles Feb 14, 2020
81f3b25
Address comments
Hexcles Mar 5, 2020
de0cb37
Update rfcs/quic.md
Hexcles Mar 11, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
115 changes: 115 additions & 0 deletions rfcs/quic.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,115 @@
# RFC 42: Add an optional QUIC server to wpt

## Summary

Add an *optional*, *off-by-default* [QUIC](https://quicwg.org/) server to wpt,
implemented with [aioquic](https://github.com/aiortc/aioquic) in Python 3. This
server is used to test web APIs that rely on the QUIC protocol (e.g.
[WebTransport](https://wicg.github.io/web-transport/)). If the server is not
enabled, `wptrunner` will skip the tests that rely on QUIC.

QUIC custom handlers use a new filename flag `.quic` (`.quic.py`). And a meta
tag is included by tests that require the QUIC server for easy filtering.

This RFC focuses on the high-level architecture and dependency management to
enable prototyping the QUIC server. The server APIs are not yet finalized, but
we commit to filing another RFC to discuss and finalize the API design once it
is ready.

See the [original issue][original-issue] for more background.

## Details

### Requirements

1. The QUIC server requires decent cryptography support; pure Python isn't
sufficient.
2. The server needs to be scriptable, and the custom handlers should live in
the wpt repo next to test files, so that they can be updated together.
3. An async programming model is preferred.

### Architecture

The QUIC server has a standalone CLI, and is managed by `wpt serve` as a
subprocess. Configurations are stored in `config.json` (same as `wptserve`), and
`wpt serve` is responsible for passing the relevant configurations (e.g. port
numbers) to the QUIC server via command line flags. `wpt serve` handles the
error gracefully if the QUIC server cannot be started.

The QUIC server is written in Python 3 and makes no attempt to support Python 2.
Its entrypoint will have a shebang that explicitly requires `python3`. An
implication is that the QUIC server will be relatively self contained, without
dependencies on other modules in wpt that are currently Python 2-only.

Code for the QUIC server itself will live in `tools/quic`.

### User interface

For **test writers**: the QUIC server will support Python custom handlers
similar to `wptserve`. The APIs are not yet finalized and will likely be
different. To more easily distinguish the two, we add a filename flag to QUIC
custom handlers (`.quic.py`). These handlers will be ignored by `wptserve`.
Tests that require the QUIC server need to be marked with a meta tag
(tentatively `<meta name="quic" content="true">` and `// META: quic=true`). This
information will be extracted by `wpt manifest` into `MANIFEST.json`, so that
`wptrunner` can skip the tests that require the QUIC server easily. A lint rule
will be added to require this meta tag if `.quic.py` appears in the file, even
though it cannot prevent all accidental dependency of the QUIC server (e.g.
through a resource script).

For **test runners**: the goal is to not introduce any observable changes to
users not yet concerned with QUIC or WebTransport. All existing tests will
continue to run without any infra changes, and all new QUIC-dependent tests will
be skipped by default. To enable the QUIC server, users need to pass a flag to
`wpt run` (or `wpt serve`). `wpt` front-end will then attempt to download the
the dependencies (wheels) from PyPI into `_venv`; maintainers of infrastructures
where PyPI cannot be used need to manage dependencies themselves (see the next
section).

### Notable dependencies

* [sys] Python 3 (>=3.6)
* [py] aioquic
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we happy with the mainteinance story of this library? How did you decide that it has the features we need to give the low-level protocol access that's required for writing tests?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@yutakahirano knows more details here.

* [py] cryptography
* [sys] openssl (>=1.0.1, already required by [HTTP/2](http2.md))
* [py] pylsqpack

All of the Python dependencies above have native extensions, which requires
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So it seems like updating the version here is going to be an infra-breaking change for vendors because work will be required on the vendor side to update to the new version (e.g. updating the deps file or uploading the wheel to the internal pypi mirror or whatever each vendor does). That seems like a pretty big concern to me. If we did vendor the source, we could have logic like "check if the correct version is available, or pip install, or build from source". Maybe there are other possibilities to address this?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm thinking about this from two perspectives:

  • For average human users, it's most likely they are using a platform supported by PyPI wheels, and the dependencies will be installed into _venv using pip automatically. Vendoring the source isn't very useful for them.
  • For browser infras, AFAIK both Firefox and Chromium discourage or even disallow building a native Python extension at run time; these dependencies will have to come from a PyPI-like repo. If that's the case, vendoring the source doesn't seem helpful, either.

I agree that this will incur a maintenance burden on the vendors if they want to run QUIC tests. (I'll acknowledge this in the risks section.) Meanwhile, I don't see vendoring the source code would help the browser vendors if my understanding above is correct -- someone would still need to build and distribute the new version.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think our test environment forbids building a native extension, although it also seems likely that the required libraries aren't available. What would definitely be possible (although not tivial) is adding the build step to the Firefox build and distributing the resulting package to the test machines; we already have various test-only binaries that need to be produced.

What worries me about the proposed setup is that updating the library version is a sync-blocking change for Gecko since we're a) using the tools/ directory directly and b) don't allow PyPI access from test machines for robustness reasons.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's true for Chromium, too. I imagine that in the future when we need to update these native dependencies, we will need to create a PR that updates the requirements.txt for venv, and ask each vendor to prepare the packages in their infras, before merging the PR. Native dependencies are difficult to deal with in general; I'm intentionally making some tradeoff here between ongoing maintenance costs and complexity.

Do you have a more concrete alternative proposal? If you think vendoring the source into the codebase is easier for Firefox infra, then I'd be happy to add that as a fallback to wheels, too. I just didn't want to vendor them if it wouldn't be used by anyone.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that adding the source would in principle be better for Mozilla, since we could in theory add it to the build system. In practice I don't know that we'd actually get around to doing that :/ It probably depends on how hard it is vs doing manual updates. But I think I'd like us to do it anyway.

In any case I think the process probably demands an RFC for version updates, since it's a change that affects multiple vendors. That's pretty heavyweight, but this really is the kind of change that can break integrations.

users to use a platform supported by the prebuilt wheels from PyPI or build and
distribute the wheels themselves (e.g. in Chromium through
[cipd](https://chromium.googlesource.com/chromium/src/+/master/docs/cipd.md)).

The source code of all dependencies will also be vendored into
`tools/third_party` to allow building from source if needed, even though we
there is no plan for that yet.

All Python dependencies are BSD-licensed.

### Alternatives considered

We have also [considered][original-issue] writing the QUIC server in other
languages (e.g. Go) and publishing a statically linked binary that would be
downloaded by `wpt` similar to how WebDriver binaries are installed. This would
incur a smaller overhead on the wpt repo in terms of new dependencies, but would
make writing [scriptable custom handlers](#requirements) harder -- we would need
to build an interface between the server and the handler scripts (which will
likely be in a different language, like Python).

Another alternative that avoids mixing Python 2 and 3 is to migrate the whole
`wptserve` to Python 3, but that work has a much bigger scope.

## Risks

The change required in `wpt` is limited, and the QUIC server itself is
standalone, so the maintenance burden of the code base will be low. The
Ecosystem Infra team at Google will be responsible for maintaining this
integration point.

A new meta tag means a new field in the manifest JSON (only needed for tests
that use the QUIC server), which should be backward compatible.

The biggest concern moving forward is the management of the binary dependencies.
Whenever a dependency is updated, we will need to notify all vendors who opt
into the QUIC server so that they can prepare the new wheels in their infras.

[original-issue]: https://github.com/web-platform-tests/wpt/issues/19114