upate tendermint to 0.40.0 #4963

SuperFluffy · 2024-12-16T14:05:01Z

Describe your changes

This patch started out as an attempt to penumbra to use crates.io cnidarium, but quickly snowballed from there to include several connected changs (primarily due to a hard coupling with prost and/or tonic):

[email protected]
[email protected]
[email protected] (and related crates such as tonic_build)
[email protected] (and related crates such as prost_build)
[email protected] (and related crates such as pbjson_build)
[email protected] (induced by the update to tonic)
[email protected]
[email protected] (induced by axum and tonic)
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected] (and related utilities such as metrics-exporter-prometheus and metrics-tracing-context)

Checklist before requesting a review

I have added guiding text to explain how a reviewer should test these changes.
If this code contains consensus-breaking changes, I have added the "consensus-breaking" label. Otherwise, I declare my belief that there are not consensus-breaking changes, for the following reason:

REPLACE THIS TEXT WITH RATIONALE (CAN BE BRIEF)

SuperFluffy · 2024-12-16T14:07:19Z

Cargo.toml

-tonic                            = { git = "https://github.com/penumbra-zone/tonic.git", tag = "v0.10.3-penumbra" }
-tonic-reflection                 = { git = "https://github.com/penumbra-zone/tonic.git", tag = "v0.10.3-penumbra" }
-tonic-web                        = { git = "https://github.com/penumbra-zone/tonic.git", tag = "v0.10.3-penumbra" }
+tendermint = { git = "https://github.com/informalsystems/tendermint-rs/", rev = "refs/pull/1480/head" }


Merged, I shall point them to their main repo. We should create an issue to followup with their patch release.

SuperFluffy · 2024-12-16T14:07:35Z

crates/bin/pclientd/src/lib.rs

@@ -344,7 +344,7 @@ impl Opt {
                            .register_encoded_file_descriptor_set(
                                penumbra_proto::FILE_DESCRIPTOR_SET,
                            )
-                            .build()
+                            .build_v1()


build() was deprecated

SuperFluffy · 2024-12-16T14:08:38Z

crates/bin/pclientd/src/proxy.rs

 use tower::ServiceExt;

 fn proxy(
    channel: Channel,
-    req: http::Request<Body>,
+    req: http::Request<BoxBody>,


Body is now a trait - BoxBody seems to have taken its place entirely.

SuperFluffy · 2024-12-16T14:09:11Z

crates/bin/pclientd/src/proxy.rs

 ) -> Pin<Box<dyn Future<Output = Result<http::Response<BoxBody>, Infallible>> + Send + 'static>> {
    tracing::debug!(headers = ?req.headers(), "proxying request");
-    // Convert request types
-    let req = req.map(|b| {


This conversion seems no longer necessary because BoxBody is an aliaes for UnsyncBoxBody<Bytes, Status>

SuperFluffy · 2024-12-16T14:09:32Z

crates/bin/pclientd/src/proxy.rs


    let rsp = channel.oneshot(req);

    async move {
        // Once we get the response, we need to convert any transport errors into
        // an Ok(HTTP response reporting an internal error), so we can have Error = Infallible
        let rsp = match rsp.await {
-            Ok(rsp) => rsp.map(|b| {


Similar to above, this conversion is no longer necessary due to the use of BoxBody

SuperFluffy · 2024-12-16T14:09:46Z

crates/bin/pclientd/src/proxy.rs

-            }),
-            Err(e) => tonic::Status::internal(format!("grpc proxy error: {e}")).to_http(),
+            Ok(rsp) => rsp,
+            Err(e) => tonic::Status::internal(format!("grpc proxy error: {e}")).into_http(),


to_http is now into_http

SuperFluffy · 2024-12-16T14:10:15Z

crates/bin/pclientd/src/proxy.rs

All changes in this file are primarily due to Body now being a traid, with tonic::body::BoxBody seemingly having taken over its place.

SuperFluffy · 2024-12-16T14:11:32Z

crates/bin/pd/Cargo.toml

@@ -28,7 +28,7 @@ anyhow                           = { workspace = true }
 ark-ff                           = { workspace = true, default-features = true }
 async-stream                     = { workspace = true }
 async-trait                      = { workspace = true }
-axum                             = "0.6"
+axum                             = { workspace = true, features = ["http2"] }


I am not completely sure about axum, axum::serve, and when/how to activate http2. I have left a lengthy comment in penumbra-core::rpc to that effect.

SuperFluffy · 2024-12-16T14:11:52Z

crates/bin/pd/src/main.rs

-            let grpc_server = penumbra_app::rpc::router(&storage, tm_proxy, enable_expensive_rpc)?;
+
+            let grpc_routes = penumbra_app::rpc::routes(&storage, tm_proxy, enable_expensive_rpc)?;
+            // let grpc_server = penumbra_app::rpc::router(&storage, tm_proxy, enable_expensive_rpc)?;


Remove commented out code

SuperFluffy · 2024-12-16T14:13:01Z

crates/core/app/src/rpc.rs

-        // typically uses HTTP/2, which requires HTTPS. Accepting HTTP/2
-        // allows local applications such as web browsers to talk to pd.
-        .accept_http1(true)
+


This still needs to be addressed.

The http2-bit might just magically work at all callsites due to our use of axum::serve. But a new trace layer might need be established, for example using tower-http.

SuperFluffy · 2024-12-16T14:13:09Z

crates/core/app/src/rpc.rs

@@ -125,8 +143,7 @@ pub fn router(
        ))))
        .add_service(we(tonic_reflection::server::Builder::configure()
            .register_encoded_file_descriptor_set(penumbra_proto::FILE_DESCRIPTOR_SET)
-            .build()
+            .build_v1()


SuperFluffy · 2024-12-16T14:14:30Z

crates/core/app/src/server/consensus.rs

@@ -25,7 +25,10 @@ fn trace_events(events: &[Event]) {
        let span = tracing::debug_span!("event", kind = ?event.kind);
        span.in_scope(|| {
            for attr in &event.attributes {
-                tracing::debug!(k = ?attr.key, v=?attr.value);
+                tracing::debug!(


These seemed to be reasonable. Tendermint no longer guarantees that keys and values are utf8: https://docs.rs/tendermint/0.40.0/tendermint/abci/enum.EventAttribute.html

SuperFluffy · 2024-12-16T14:15:22Z

crates/core/app/src/server/events.rs

+                //      plain bytes and need not be utf8. This messes with the
+                //      regex and is likely not desirable to have in downstream
+                //      consumers of the indexed events.
+                match attr.key_str() {


Related to tendermint-rs no longer guaranteeing that keys are utf8: https://docs.rs/tendermint/0.40.0/tendermint/abci/enum.EventAttribute.html

It seemed to make sense that, in the case of keys not being utf8, we do not want to index them. But should this case just be dropped quietly?

SuperFluffy · 2024-12-16T14:15:48Z

crates/core/app/src/server/info.rs

@@ -259,6 +259,7 @@ impl Info {
                        channel_id: chan_id,
                        port_id: PortId::transfer(),
                        channel_end: channel,
+                        upgrade_sequence: 0,


New field; I did not know what to do with it, so opted for the default value (as per protobuf convention).

SuperFluffy · 2024-12-16T14:15:54Z

crates/core/app/src/server/info.rs

@@ -308,6 +309,7 @@ impl Info {
                            channel_id: chan_id,
                            port_id: PortId::transfer(),
                            channel_end: channel,
+                            upgrade_sequence: 0,


New field; I did not know what to do with it, so opted for the default value (as per protobuf convention).

SuperFluffy · 2024-12-16T14:16:34Z

crates/core/component/ibc/src/component/msg_handler/channel_close_confirm.rs

@@ -70,6 +70,7 @@ impl MsgHandler for MsgChannelCloseConfirm {
            remote: expected_counterparty,
            connection_hops: expected_connection_hops,
            version: channel.version.clone(),
+            ..ChannelEnd::default()
