Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

io_uring: update API and docs #38677

Merged
merged 8 commits into from
Mar 8, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -16,30 +16,31 @@ option (udpa.annotations.file_status).package_version_status = ACTIVE;

// Configuration for default socket interface that relies on OS dependent syscall to create
// sockets.
// [#next-free-field: 6]
message DefaultSocketInterface {
// Enable io_uring for socket operations if the kernel supports. io_uring is only valid in
// Linux with at least kernel version 5.6. Otherwise, Envoy will fall back to use the default
// socket operations. The default is false.
bool enable_io_uring = 1;
// io_uring options. io_uring is only valid in Linux with at least kernel version 5.11. Otherwise,
// Envoy will fall back to use the default socket API. If not set then io_uring will not be
// enabled.
IoUringOptions io_uring_options = 1;
}

message IoUringOptions {
// The size for io_uring submission queues (SQ). io_uring is built with a fixed size in each
// thread during configuration, and each io_uring operation creates a submission queue
// entry (SQE). The default is 1000.
google.protobuf.UInt32Value io_uring_size = 2;
google.protobuf.UInt32Value io_uring_size = 1;

// Enable io_uring submission queue polling (SQPOLL). io_uring SQPOLL mode polls all SQEs in the
// SQ in the kernel thread. io_uring SQPOLL mode may reduce latency and increase CPU usage as a
// cost. The default is false.
bool enable_io_uring_submission_queue_polling = 3;
bool enable_submission_queue_polling = 2;

// The size of an io_uring socket's read buffer. Each io_uring read operation will allocate a
// buffer of the given size. If the given buffer is too small, the socket will have read multiple
// times for all the data. The default is 8192.
google.protobuf.UInt32Value io_uring_read_buffer_size = 4;
google.protobuf.UInt32Value read_buffer_size = 3;

// The write timeout of an io_uring socket on closing in ms. io_uring writes and closes
// asynchronously. If the remote stops reading, the io_uring write operation may never complete.
// The operation is canceled and the socket is closed after the timeout. The default is 1000.
google.protobuf.UInt32Value io_uring_write_timeout_ms = 5;
google.protobuf.UInt32Value write_timeout_ms = 4;
}
4 changes: 2 additions & 2 deletions changelogs/current.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -234,7 +234,7 @@ new_features:
``envoy.tcp_proxy.receive_before_connect``.
- area: sockets
change: |
Added :ref:`enable_io_uring <envoy_v3_api_field_extensions.network.socket_interface.v3.DefaultSocketInterface.enable_io_uring>` to
support io_uring.
Added an :ref:`io_uring <envoy_v3_api_field_extensions.network.socket_interface.v3.DefaultSocketInterface.io_uring_options>` option in
default socket interface to support io_uring.

deprecated:
49 changes: 49 additions & 0 deletions docs/root/configuration/other_features/_include/io_uring.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
static_resources:
listeners:
- address:
socket_address:
address: 0.0.0.0
port_value: 80
filter_chains:
- filters:
- name: envoy.filters.network.http_connection_manager
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
codec_type: AUTO
stat_prefix: ingress_http
route_config:
name: local_route
virtual_hosts:
- name: app
domains:
- "*"
routes:
- match:
prefix: "/"
route:
cluster: local_service
http_filters:
- name: envoy.filters.http.router
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router

clusters:
- name: local_service
type: STRICT_DNS
lb_policy: ROUND_ROBIN
load_assignment:
cluster_name: local_service
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: 127.0.0.1
port_value: 8080

bootstrap_extensions:
- name: envoy.extensions.network.socket_interface.default_socket_interface
typed_config:
"@type": type.googleapis.com/envoy.extensions.network.socket_interface.v3.DefaultSocketInterface
io_uring_options: {}
default_socket_interface: "envoy.extensions.network.socket_interface.default_socket_interface"
35 changes: 35 additions & 0 deletions docs/root/configuration/other_features/io_uring.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
.. _config_io_uring:

io_uring
========

* :ref:`v3 API reference <envoy_v3_api_msg_extensions.network.socket_interface.v3.IoUringOptions>`

.. attention::

io_uring is experimental and is currently under active development.

.. note::

This feature is not supported on Windows.

io_uring is an API for asynchronous I/O available in modern Linux kernels, designed to reduce system
calls in network I/O and enhance performance. Envoy can be configured to use io_uring for all TCP
listeners and connections.

To enable io_uring in Envoy, the Linux kernel must be at least version 5.11.

Example Configuration
---------------------

.. literalinclude:: _include/io_uring.yaml
:language: yaml
:linenos:
:lines: 44-49
:caption: :download:`io_uring.yaml <_include/io_uring.yaml>`

In this configuration, io_uring is enabled in the bootstrap extension, and the default socket
interface is explicitly defined. As a result, Envoy initializes a socket interface with io_uring
support, replacing the default socket interface that uses the traditional socket API.

If the kernel does not support io_uring, Envoy will fall back to the traditional socket API.
1 change: 1 addition & 0 deletions docs/root/configuration/other_features/other_features.rst
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ Other features
hyperscan
internal_listener
rate_limit
io_uring
vcl
wasm
wasm_service
Expand Down
11 changes: 6 additions & 5 deletions source/common/network/socket_interface_impl.cc
Original file line number Diff line number Diff line change
Expand Up @@ -171,13 +171,14 @@ Server::BootstrapExtensionPtr SocketInterfaceImpl::createBootstrapExtension(
const auto& message = MessageUtil::downcastAndValidate<
const envoy::extensions::network::socket_interface::v3::DefaultSocketInterface&>(
config, context.messageValidationVisitor());
if (message.enable_io_uring() && Io::isIoUringSupported()) {
if (message.has_io_uring_options() && Io::isIoUringSupported()) {
const auto& options = message.io_uring_options();
std::shared_ptr<Io::IoUringWorkerFactoryImpl> io_uring_worker_factory =
std::make_shared<Io::IoUringWorkerFactoryImpl>(
PROTOBUF_GET_WRAPPED_OR_DEFAULT(message, io_uring_size, 1000),
message.enable_io_uring_submission_queue_polling(),
PROTOBUF_GET_WRAPPED_OR_DEFAULT(message, io_uring_read_buffer_size, 8192),
PROTOBUF_GET_WRAPPED_OR_DEFAULT(message, io_uring_write_timeout_ms, 1000),
PROTOBUF_GET_WRAPPED_OR_DEFAULT(options, io_uring_size, 1000),
options.enable_submission_queue_polling(),
PROTOBUF_GET_WRAPPED_OR_DEFAULT(options, read_buffer_size, 8192),
PROTOBUF_GET_WRAPPED_OR_DEFAULT(options, write_timeout_ms, 1000),
context.threadLocal());
io_uring_worker_factory_ = io_uring_worker_factory;

Expand Down
7 changes: 4 additions & 3 deletions test/integration/socket_interface_integration_test.cc
Original file line number Diff line number Diff line change
Expand Up @@ -25,15 +25,16 @@ class SocketInterfaceIntegrationTest : public BaseIntegrationTest,

static std::string config(bool enable_io_uring = false) {
// At least one empty filter chain needs to be specified.
return absl::StrCat(echoConfig(), absl::StrFormat(R"EOF(
return absl::StrCat(echoConfig(),
absl::StrFormat(R"EOF(
bootstrap_extensions:
- name: envoy.extensions.network.socket_interface.default_socket_interface
typed_config:
"@type": type.googleapis.com/envoy.extensions.network.socket_interface.v3.DefaultSocketInterface
enable_io_uring: %s
%s
default_socket_interface: "envoy.extensions.network.socket_interface.default_socket_interface"
)EOF",
enable_io_uring ? "true" : "false"));
enable_io_uring ? "io_uring_options: {}" : ""));
}
static std::string echoConfig() {
return absl::StrCat(ConfigHelper::baseConfig(), R"EOF(
Expand Down
Loading