diff --git a/docs/neutron/modules/contract-manager/client.md b/docs/neutron/modules/contract-manager/client.md index 0ec2710b2..c433ad53a 100644 --- a/docs/neutron/modules/contract-manager/client.md +++ b/docs/neutron/modules/contract-manager/client.md @@ -7,20 +7,20 @@ In this section we describe the queries required on grpc server. ```protobuf // Query defines the gRPC querier service. service Query { - // Parameters queries the parameters of the module. - rpc Params(QueryParamsRequest) returns (QueryParamsResponse) { - option (google.api.http).get = "/neutron-org/neutron/contractmanager/params"; - } - - // Queries a Failures by address. - rpc Failure(QueryGetFailuresByAddressRequest) returns (QueryGetFailuresByAddressResponse) { - option (google.api.http).get = "/neutron-org/neutron/contractmanager/failure/{address}"; - } - - // Queries a list of failed addresses. - rpc AllFailures(QueryAllFailureRequest) returns (QueryAllFailureResponse) { - option (google.api.http).get = "/neutron-org/neutron/contractmanager/failure"; - } + // Parameters queries the parameters of the module. + rpc Params(QueryParamsRequest) returns (QueryParamsResponse) { + option (google.api.http).get = "/neutron-org/neutron/contractmanager/params"; + } + + // Queries a Failures by address. + rpc Failure(QueryGetFailuresByAddressRequest) returns (QueryGetFailuresByAddressResponse) { + option (google.api.http).get = "/neutron-org/neutron/contractmanager/failure/{address}"; + } + + // Queries a list of failed addresses. + rpc AllFailures(QueryAllFailureRequest) returns (QueryAllFailureResponse) { + option (google.api.http).get = "/neutron-org/neutron/contractmanager/failure"; + } } ``` @@ -43,22 +43,23 @@ neutrond query contractmanager failures Output: ```yaml - failures: - - address: neutron1nc5tatafv6eyq7llkr2gv50ff9e22mnf70qgjlv737ktmt4eswrqcd0mrx - id: 0 - sudo_payload: - - address: neutron14hj2tavq8fpesdwxxcu44rty3hh90vhujrvcmstl4zr3txmfvw9s5c2epq - id: 1 - sudo_payload: - pagination: - next_key: null - total: "2" +failures: +- address: neutron1nxshmmwrvxa2cp80nwvf03t8u5kvl2ttr8m8f43vamudsqrdvs8qqvfwpj + error: 'codespace: wasm, code: 5' + id: "1" + sudo_payload: +- address: neutron1nxshmmwrvxa2cp80nwvf03t8u5kvl2ttr8m8f43vamudsqrdvs8qqvfwpj + error: 'codespace: wasm, code: 5' + id: "2" + sudo_payload: +- address: neutron1nxshmmwrvxa2cp80nwvf03t8u5kvl2ttr8m8f43vamudsqrdvs8qqvfwpj + error: 'codespace: contractmanager, code: 1103' + id: "3" + sudo_payload: ``` - - Returns list of all failures for specific contract address. ```bash @@ -70,22 +71,59 @@ neutrond query contractmanager failures [address] Returns failures for specific contract address: ```shell - neutrond query contractmanager failures neutron14hj2tavq8fpesdwxxcu44rty3hh90vhujrvcmstl4zr3txmfvw9s5c2epq + neutrond query contractmanager failures neutron1nxshmmwrvxa2cp80nwvf03t8u5kvl2ttr8m8f43vamudsqrdvs8qqvfwpj + ``` + +Output: + + ```yaml +failures: +- address: neutron1nxshmmwrvxa2cp80nwvf03t8u5kvl2ttr8m8f43vamudsqrdvs8qqvfwpj + error: 'codespace: wasm, code: 5' + id: "1" + sudo_payload: +- address: neutron1nxshmmwrvxa2cp80nwvf03t8u5kvl2ttr8m8f43vamudsqrdvs8qqvfwpj + error: 'codespace: wasm, code: 5' + id: "2" + sudo_payload: +- address: neutron1nxshmmwrvxa2cp80nwvf03t8u5kvl2ttr8m8f43vamudsqrdvs8qqvfwpj + error: 'codespace: contractmanager, code: 1103' + id: "3" + sudo_payload: + ``` + + + +### Failure Details + +Returns an exact error why contract failed to process certain ibc acknowledgement. + +> **Note** +> It is a CLI-like query, and you can not perform it onchain, e.g. you can not make a query from a contract. The reason is - cosmos-sdk do not store raw errors in a storage due to non-deterministic nature of errors. You can not consider the data which are not under you control as deterministic. But to make developers life easier we inorporated the following mechanism. In case of error happened during the `sudo` call we emit an event with full error text under the key `(contract_address,failure_id)`. Events are not the part of the consensus, and it's safe to emit any data you want. The CLI command looks for [transaction](https://github.com/neutron-org/neutron/blob/v2.0.2/x/contractmanager/client/cli/query_failure.go#L85) by a set of events. +> +> **Note** +> If the node you are making query to either does not index transaction or already cleared the block with the wanted transaction, you get the error - `detailed failure error message not found in node events`. In this case you need to query a node, which: +> +> 1) Indexes transactions. +> +> 2) Keeps block with wanted height. + +```shell +neutrond q contractmanager failure-details [address] [failure_id] +``` + +
+ Example + Query the detailed error related to a contract's sudo call failure: + + ```shell + neutrond q contractmanager failure-details neutron1nxshmmwrvxa2cp80nwvf03t8u5kvl2ttr8m8f43vamudsqrdvs8qqvfwpj 1 ``` Output: ```yaml - failures: - - address: neutron14hj2tavq8fpesdwxxcu44rty3hh90vhujrvcmstl4zr3txmfvw9s5c2epq - id: 0 - sudo_payload: - - address: neutron14hj2tavq8fpesdwxxcu44rty3hh90vhujrvcmstl4zr3txmfvw9s5c2epq - id: 1 - sudo_payload: - pagination: - next_key: null - total: "2" +dispatch: submessages: Generic error: Integrations test mock submsg error: execute wasm contract failed ```
diff --git a/docs/neutron/modules/contract-manager/overview.md b/docs/neutron/modules/contract-manager/overview.md index 8e96e276e..248fa20ce 100644 --- a/docs/neutron/modules/contract-manager/overview.md +++ b/docs/neutron/modules/contract-manager/overview.md @@ -15,13 +15,29 @@ But this in turn exposes the problem of informing the owner of the contract that To ensure that the state of the contract is consistent, the call to the sudo handler takes place in a temporary state (using `CacheContext`), which is written to the active state if the call succeeds. +## SudoLimitWrapper + +[SudoLimitWrapper](https://github.com/neutron-org/neutron/blob/v2.0.0/x/contractmanager/ibc_middleware.go#L14) is a middleware wrapper with interface + +```go +type WasmKeeper interface { + HasContractInfo(ctx sdk.Context, contractAddress sdk.AccAddress) bool + Sudo(ctx sdk.Context, contractAddress sdk.AccAddress, msg []byte) ([]byte, error) +} +``` + +It performs two important functions: + +1. Make sure that the sudo contract call does not use more gas than allowed by a chain [parameter](#gas-limitation). If the gas limit is exceeded and an out of gas panic occurs inside the sudo call, the wrapper intercepts the panic (only the out of gas panic is intercepted) and converts it to an error. +2. Capture an error from the sudo handler of the interchaintxs module, either directly initiated by the contract or an error that was received from an out of gas panic and write the error with the data to [Failures](#failures-details) for further processing + ### Gas limitation To make sure there are no exploits with infinite recursion of IBC messages which call other IBC messages in sudo handler we use constant gas `LIMIT` to spend. The LIMIT is small, so you can't place extensive work in Sudo handlers. As a workaround, in such cases you can use Sudo handlers to simply store required payload in contract's state, and use Execute messages to handle results separately. If your contract exceeds this constant `LIMIT`, it will terminate sudo handler call and save a `Failure` with full call info. You can [resubmit failure](#resubmitfailure) from this contract. -The `LIMIT` is defined by `SudoCallGasLimit` module's parameter. +The `LIMIT` is defined by [`SudoCallGasLimit`](https://github.com/neutron-org/neutron/blob/v2.0.2/x/contractmanager/types/params.pb.go#L29) module's parameter. We provide an ability to resubmit bindings through the contract that initiated the IBC transaction. @@ -67,6 +83,21 @@ Response: type FailuresResponse struct { Failures []contractmanagertypes.Failure `json:"failures"` } + +type Failure struct { + // Address of the failed contract + Address string `protobuf:"bytes,1,opt,name=address,proto3" json:"address,omitempty"` + // Id of the failure under specific address + Id uint64 `protobuf:"varint,2,opt,name=id,proto3" json:"id,omitempty"` + // Serialized MessageSudoCallback with Packet and Ack(if exists) + SudoPayload []byte `protobuf:"bytes,3,opt,name=sudo_payload,json=sudoPayload,proto3" json:"sudo_payload,omitempty"` + // Redacted error response of the sudo call. Full error is emitted as an event + Error string `protobuf:"bytes,4,opt,name=error,proto3" json:"error,omitempty"` +} ``` You're encouraged to use our neutron-sdk implementation in contracts - [request](https://github.com/neutron-org/neutron-sdk/blob/feat/contract-manager-resubmit/packages/neutron-sdk/src/bindings/query.rs#L61) and [response](https://github.com/neutron-org/neutron-sdk/blob/feat/contract-manager-resubmit/packages/neutron-sdk/src/bindings/query.rs#L119) + +## Failures details + +In the case of a contract-initiated error, the error is stored in [Failure](state.md). Since raw errors are not part of the consensus and cannot guarantee determinism when saved to the state, errors saved in `Failure` are redacted to `codespace + code_id` using the [RedactError](https://github.com/neutron-org/neutron/blob/v2.0.2/x/contractmanager/keeper/failure.go#L109) function. At the same time, we emit the full text of the error into transaction events that do not affect the consensus and if you need to get detailed information about the error returned by the contract, you need to find the transaction in which the redacted error occurred and see the events in which we emit the error. To simplify this procedure we added a special [cli query](./client.md#failure-details), the error can be found by the `Failure.Address` and the `Failure.Id`. diff --git a/docs/neutron/modules/contract-manager/state.md b/docs/neutron/modules/contract-manager/state.md index 4bb92a984..27c0df26f 100644 --- a/docs/neutron/modules/contract-manager/state.md +++ b/docs/neutron/modules/contract-manager/state.md @@ -1,4 +1,9 @@ # State -The ContractManager module stores one [Failure](https://github.com/neutron-org/neutron/blob/v1.0.4/proto/contractmanager/genesis.proto#L12) per contract address and record id. -`Failure` contains all the necessary info to store data about ACK sudo handler call failure. `address` contains contract address and it is used in conjunction with `id` field to create index of the record in the `KVStore`. `ack_id` is used to identify ACK request that was failed, `ack_type` can take two values : `ack`, and `timeout`. +The ContractManager module stores [Failure](https://github.com/neutron-org/neutron/blob/v2.0.0/proto/neutron/contractmanager/failure.proto#L11) under `contract address` and `record id` key. +`Failure` contains all the necessary info about ACK sudo handler call failure. + +- `address` contains contract address and it is used in conjunction with +- `id` field to create index of the record in the `KVStore`. +- `SudoPayload` - serialized MessageSudoCallback with Packet and Ack(if exists) +- `Error` - redacted error response of the sudo call. Full error is emitted as an event diff --git a/docs/neutron/modules/interchain-txs/overview.md b/docs/neutron/modules/interchain-txs/overview.md index 9f9a42034..9151912bd 100644 --- a/docs/neutron/modules/interchain-txs/overview.md +++ b/docs/neutron/modules/interchain-txs/overview.md @@ -25,7 +25,7 @@ complete list of IBC events for each module message in the [messages](./messages ## Sudo errors handling -Interchaintxs module configured the following way, all the errors from a sudo handler are being suppressed by [contract manager middleware](/neutron/modules/contract-manager/overview#concepts), sudo handler is limited with [LIMIT](/neutron/modules/contract-manager/overview#gas-limitation) amount of gas +Interchaintxs module configured the following way, all the errors from a sudo handler are being suppressed by [contract manager middleware](/neutron/modules/contract-manager/overview#sudolimitwrapper), sudo handler is limited with [LIMIT](/neutron/modules/contract-manager/overview#gas-limitation) amount of gas ## Importing interchaintxs module @@ -48,6 +48,74 @@ If you use interchaintxs module in your application and if your Sudo handler fai > execute [RegisterInterchainAccount message]( /neutron/modules/interchain-txs/messages#msgregisterinterchainaccount) again to > recover access to your interchain account. > **Note** Keep in mind, new channel is created +> + +## Sudo Handlers + +The interchaintxs module in neutrond configured the following way. + +Wasmd Sudo handler wrapped with [SudoLimitWrapper](../contract-manager/overview.md#sudolimitwrapper) + +And acknoledgement packet follows the way +`OnAcknowledgementPacket` --> [SudoLimitWrapper](../contract-manager/overview.md#sudolimitwrapper) --> `Wasmd Sudo handler` + +Using `SudoLimitWrapper` has two purposes: + +1. Suppress the sudo handler error itself, and mark the ibc acknowledgement packet as received and processed. Other way, the error makes relayer send an acknowledgement again and again. Information about an unsuccessfully processed ack is stored in [state](../contract-manager/state.md). +2. Limit the amount of gas available for sudo handler execution. Out of gas panic will later be captured by `SudoLimitWrapper` and converted into an error. + +## Failed interchain txs + +Not every interchaintx executes succesfully on a remote network. Some of them fail to execute with errors and then you get ibc acknowledgement with `Error` type. The error is passed into the caller contract via sudo call with SudoMsg::Error [variant](../../../tutorials/cosmwasm_ica.md#ibc-events) + +Unfortunately, to avoid the nondeterminism associated with error text generation, the error text is severely truncated by [redact down](https://github.com/cosmos/ibc-go/blob/v7.3.1/modules/apps/27-interchain-accounts/host/ibc_module.go#L115) to the error code without any additional details, before converting into AcknowledgementError. + +Find the error text is possible if host chain includes ibc-go v7.2.3+, v7.3.2+, v8.0.1+, v8.1+ which include patch [5541](https://github.com/cosmos/ibc-go/pull/5541) +` q interchain-accounts host packet-events ` + +Where: + +- `binary` is a binary on the chain you are working with (the remote chain) +- `seq-id` - sequence ID of the IBC message sent to the remote chain. The seq-id is returned to the contract in the [SubmitTx](./messages.md#response) response +- `channel-id` is the ID of the ICA's channel on the remote chain's side. You should know it from registration [procedure](../../../tutorials/cosmwasm-ica.md#2-register-an-interchain-account) via `SudoMsg::OpenAck` from `counterparty_channel_id` field. If you missed it you can always get counterparty channel-id with CLI command `neutrond q ibc channel end ` +- `src-channel-id` is the channel you intechain account associated with. +- `src-port` is the port you interchain account is associated with. +You should know both `src-channel-id` and `src-port` from registration [procedure](../../../tutorials/cosmwasm-ica.md#2-register-an-interchain-account). Also `src-port` is `icacontroller-.` where `ica_id` defined by you during ica registration. + +Output example (filtered events): + +```json +{ +"type": "ibccallbackerror-ics27_packet", +"attributes": [ + { + "key": "ibccallbackerror-module", + "value": "interchainaccounts", + "index": true + }, + { + "key": "ibccallbackerror-host_channel_id", + "value": "channel-2", + "index": true + }, + { + "key": "ibccallbackerror-success", + "value": "false", + "index": true + }, + { + "key": "ibccallbackerror-error", + "value": "invalid validator address: decoding bech32 failed: invalid separator index -1: invalid address", + "index": true + } +] +} +``` + +On earlier versions of ibc-go it's barely possible to get full text error due to [patch](https://github.com/cosmos/ibc-go/commit/fdbb508c1ca68811206d7175fb9e202c1611a43e). + +In the IBC error acknowledgement you get ABCI error and a code, e.g. `codespace: wasm, code: 5` +where codespace usually is `ModuleName`, and `code` is uniq code for the module. `codespace` and `code` pair uniq for the whole app. You can find the error description in source code. Usualy all the error of the module are [placed](https://github.com/CosmWasm/wasmd/blob/5f444cd9d393513e534cbfa9a0e938295c4e84e1/x/wasm/types/errors.go#L25) in `x//types/errors.go` where `module` is the module where the error was thrown ## Relaying diff --git a/versioned_docs/version-1.0/neutron/modules/interchain-txs/overview.md b/versioned_docs/version-1.0/neutron/modules/interchain-txs/overview.md index d5b386476..c4b05ce95 100644 --- a/versioned_docs/version-1.0/neutron/modules/interchain-txs/overview.md +++ b/versioned_docs/version-1.0/neutron/modules/interchain-txs/overview.md @@ -23,25 +23,89 @@ a [Sudo() call](https://github.com/CosmWasm/wasmd/blob/288609255ad92dfe5c54eae57 and a custom [message scheme](https://github.com/neutron-org/neutron/blob/v1.0.4/x/contractmanager/types/sudo.go). You can find a complete list of IBC events for each module message in the [messages](./messages) section. -> **Note**: if your Sudo handler fails, the acknowledgment won't be marked as processed inside the IBC module. This will +> **Note**: if your Sudo handler fails, the acknowledgment will be marked as processed inside the IBC module anyway. +> The module designed this way to avoid > make most IBC relayers try to submit the acknowledgment over and over again. And since the ICA channels are `ORDERED`, > ACKs must be processed in the same order as corresponding transactions were sent, meaning no further acknowledgments -> will be process until the previous one processed successfully. +> will be process until the previous one processed successfully. In addition, if sudo handler fails, we save the packetId +> failed to be processed with contract manager's failures [state](../contract-manager/state.md). It is possible to suppress +> Sudo handler fails by using [cachedContext](#sudo-handlers) with th handlers. > -> We strongly recommend developers to write Sudo handlers very carefully and keep them as simple as possible. If you do -> want to have elaborate logic in your handler, you should verify the acknowledgement data before making any state -> changes; that way you can, if the data received with the acknowledgement is incompatible with executing the handler -> logic normally, return an `Ok()` response immediately, which will prevent the acknowledgement from being resubmitted. - > **Note**: there is no dedicated event for a closed channel (ICA disables all messages related to closing the channels) > . Your channel, however, can still be closed if a packet timeout occurs; thus, if you are notified about a packet > timeout, you can be sure that the affected channel was closed. Please note that it is generally a good practice to set > the packet timeout for your interchain transactions to a really large value. > -> If the timeout occurs anyway, you can just +> If the timeout occurs anyway, you can just > execute [RegisterInterchainAccount message]( /neutron/modules/interchain-txs/messages#msgregisterinterchainaccount) again to > recover access to your interchain account. +## Sudo Handlers + +Before calling a sudo handler, we create a child context [cachedCtx](#cachedctx), with which we call this handler. +Calling a handler via a child context has 2 purposes: + +1. Suppress the sudo handler error itself, and mark the ibc acknowledgement packet as received and processed. Other way, the error makes relayer send an acknowledgement again and again. Information about an unsuccessfully processed ack is stored in [state](../contract-manager/state.md). +2. Suppress out of gas panic triggered by sudo handler operation + +### CachedCtx + +CachedCtx is created with `gas limit = gas limit in the parent context - GasReserve`, where `GasReserve` is a constant equal to 15000. +This reserve gas is needed to guarantee that [Failure](../contract-manager/state.md) is saved, for example in case a contract has used all the gas it is entitled to and ended with an out of gas panic. + +## Failed interchain txs + +Not every interchaintx executes succesfully on a remote network. Some of them fail to execute with errors and then you get ibc acknowledgement with `Error` type. The error is passed into the caller contract via sudo call with SudoMsg::Error [variant](../../../tutorials/cosmwasm_ica.md#ibc-events) + +Unfortunately, to avoid the nondeterminism associated with error text generation, the error text is severely truncated by [redact down](https://github.com/cosmos/ibc-go/blob/v7.3.1/modules/apps/27-interchain-accounts/host/ibc_module.go#L115) to the error code without any additional details, before converting into AcknowledgementError. + +Find the error text is possible if host chain includes ibc-go v7.2.3+, v7.3.2+, v8.0.1+, v8.1+ which include patch [5541](https://github.com/cosmos/ibc-go/pull/5541) +` q interchain-accounts host packet-events ` + +Where: + +- `binary` is a binary on the chain you are working with (the remote chain) +- `seq-id` - sequence ID of the IBC message sent to the remote chain. The seq-id is returned to the contract in the [SubmitTx](./messages.md#response) response +- `channel-id` is the ID of the ICA's channel on the remote chain's side. You should know it from registration [procedure](../../../tutorials/cosmwasm-ica.md#2-register-an-interchain-account) via `SudoMsg::OpenAck` from `counterparty_channel_id` field. If you missed it you can always get counterparty channel-id with CLI command `neutrond q ibc channel end ` +- `src-channel-id` is the channel you intechain account associated with. +- `src-port` is the port you interchain account is associated with. +You should know both `src-channel-id` and `src-port` from registration [procedure](../../../tutorials/cosmwasm-ica.md#2-register-an-interchain-account). Also `src-port` is `icacontroller-.` where `ica_id` defined by you during ica registration. + +Output example (filtered events): + +```json +{ +"type": "ibccallbackerror-ics27_packet", +"attributes": [ + { + "key": "ibccallbackerror-module", + "value": "interchainaccounts", + "index": true + }, + { + "key": "ibccallbackerror-host_channel_id", + "value": "channel-2", + "index": true + }, + { + "key": "ibccallbackerror-success", + "value": "false", + "index": true + }, + { + "key": "ibccallbackerror-error", + "value": "invalid validator address: decoding bech32 failed: invalid separator index -1: invalid address", + "index": true + } +] +} +``` + +On earlier versions of ibc-go it's barely possible to get full text error due to [patch](https://github.com/cosmos/ibc-go/commit/fdbb508c1ca68811206d7175fb9e202c1611a43e). + +In the IBC error acknowledgement you get ABCI error and a code, e.g. `codespace: wasm, code: 5` +where codespace usually is `ModuleName`, and `code` is uniq code for the module. `codespace` and `code` pair uniq for the whole app. You can find the error description in source code. Usualy all the error of the module are [placed](https://github.com/CosmWasm/wasmd/blob/5f444cd9d393513e534cbfa9a0e938295c4e84e1/x/wasm/types/errors.go#L25) in `x//types/errors.go` where `module` is the module where the error was thrown + ## Relaying Neutron introduces smart-contract level callbacks for IBC packets. From an IBC relayer's perspective, this means that