updated icatx docs

docs/neutron/modules/contract-manager/client.md

@@ -7,20 +7,20 @@ In this section we describe the queries required on grpc server.
 ```protobuf
 // Query defines the gRPC querier service.
 service Query {
-	// Parameters queries the parameters of the module.
-	rpc Params(QueryParamsRequest) returns (QueryParamsResponse) {
-		option (google.api.http).get = "/neutron-org/neutron/contractmanager/params";
-	}
-
-	// Queries a Failures by address.
-	rpc Failure(QueryGetFailuresByAddressRequest) returns (QueryGetFailuresByAddressResponse) {
-		option (google.api.http).get = "/neutron-org/neutron/contractmanager/failure/{address}";
-	}
-
-	// Queries a list of failed addresses.
-	rpc AllFailures(QueryAllFailureRequest) returns (QueryAllFailureResponse) {
-		option (google.api.http).get = "/neutron-org/neutron/contractmanager/failure";
-	}
+ // Parameters queries the parameters of the module.
+ rpc Params(QueryParamsRequest) returns (QueryParamsResponse) {
+  option (google.api.http).get = "/neutron-org/neutron/contractmanager/params";
+ }
+
+ // Queries a Failures by address.
+ rpc Failure(QueryGetFailuresByAddressRequest) returns (QueryGetFailuresByAddressResponse) {
+  option (google.api.http).get = "/neutron-org/neutron/contractmanager/failure/{address}";
+ }
+
+ // Queries a list of failed addresses.
+ rpc AllFailures(QueryAllFailureRequest) returns (QueryAllFailureResponse) {
+  option (google.api.http).get = "/neutron-org/neutron/contractmanager/failure";
+ }
 }
 ```
 
@@ -43,22 +43,23 @@ neutrond query contractmanager failures
 Output:
 
   ```yaml
-  failures:
-    - address: neutron1nc5tatafv6eyq7llkr2gv50ff9e22mnf70qgjlv737ktmt4eswrqcd0mrx
-      id: 0
-      sudo_payload: <serialized msg of MessageSudoCallback type>
-    - address: neutron14hj2tavq8fpesdwxxcu44rty3hh90vhujrvcmstl4zr3txmfvw9s5c2epq
-      id: 1
-      sudo_payload: <serialized msg of MessageSudoCallback type>
-  pagination:
-    next_key: null
-    total: "2"
+failures:
+- address: neutron1nxshmmwrvxa2cp80nwvf03t8u5kvl2ttr8m8f43vamudsqrdvs8qqvfwpj
+  error: 'codespace: wasm, code: 5'
+  id: "1"
+  sudo_payload: <serialized msg of MessageSudoCallback type>
+- address: neutron1nxshmmwrvxa2cp80nwvf03t8u5kvl2ttr8m8f43vamudsqrdvs8qqvfwpj
+  error: 'codespace: wasm, code: 5'
+  id: "2"
+  sudo_payload: <serialized msg of MessageSudoCallback type>
+- address: neutron1nxshmmwrvxa2cp80nwvf03t8u5kvl2ttr8m8f43vamudsqrdvs8qqvfwpj
+  error: 'codespace: contractmanager, code: 1103'
+  id: "3"
+  sudo_payload: <serialized msg of MessageSudoCallback type>
   ```
 
 </details>
 
-
-
 Returns list of all failures for specific contract address.
 
 ```bash
@@ -70,22 +71,59 @@ neutrond query contractmanager failures [address]
   Returns failures for specific contract address:
 
   ```shell
-  neutrond query contractmanager failures neutron14hj2tavq8fpesdwxxcu44rty3hh90vhujrvcmstl4zr3txmfvw9s5c2epq
+  neutrond query contractmanager failures neutron1nxshmmwrvxa2cp80nwvf03t8u5kvl2ttr8m8f43vamudsqrdvs8qqvfwpj
+  ```
+
+Output:
+
+  ```yaml
+failures:
+- address: neutron1nxshmmwrvxa2cp80nwvf03t8u5kvl2ttr8m8f43vamudsqrdvs8qqvfwpj
+  error: 'codespace: wasm, code: 5'
+  id: "1"
+  sudo_payload: <serialized msg of MessageSudoCallback type>
+- address: neutron1nxshmmwrvxa2cp80nwvf03t8u5kvl2ttr8m8f43vamudsqrdvs8qqvfwpj
+  error: 'codespace: wasm, code: 5'
+  id: "2"
+  sudo_payload: <serialized msg of MessageSudoCallback type>
+- address: neutron1nxshmmwrvxa2cp80nwvf03t8u5kvl2ttr8m8f43vamudsqrdvs8qqvfwpj
+  error: 'codespace: contractmanager, code: 1103'
+  id: "3"
+  sudo_payload: <serialized msg of MessageSudoCallback type>
+  ```
+
+</details>
+
+### Failure Details
+
+Returns an exact error why contract failed to process certain ibc acknowledgement.
+
+> **Note**
+> It is a CLI-like query, and you can not perform it onchain, e.g. you can not make a query from a contract. The reason is - cosmos-sdk do not store raw errors in a storage due to non-deterministic nature of errors. You can not consider the data which are not under you control as deterministic. But to make developers life easier we inorporated the following mechanism. In case of error happened during the `sudo` call we emit an event with full error text under the key `(contract_address,failure_id)`. Events are not the part of the consensus, and it's safe to emit any data you want. The CLI command looks for [transaction](https://github.com/neutron-org/neutron/blob/v2.0.2/x/contractmanager/client/cli/query_failure.go#L85) by a set of events.
+>
+> **Note**
+> If the node you are making query to either does not index transaction or already cleared the block with the wanted transaction, you get the error - `detailed failure error message not found in node events`. In this case you need to query a node, which:
+>
+> 1) Indexes transactions.
+>
+> 2) Keeps block with wanted height.
+
+```shell
+neutrond q contractmanager failure-details [address] [failure_id]
+```
+
+<details>
+  <summary>Example</summary>
+  Query the detailed error related to a contract's sudo call failure:
+
+  ```shell
+  neutrond q contractmanager failure-details neutron1nxshmmwrvxa2cp80nwvf03t8u5kvl2ttr8m8f43vamudsqrdvs8qqvfwpj 1
   ```
 
 Output:
 
   ```yaml
-  failures:
-    - address: neutron14hj2tavq8fpesdwxxcu44rty3hh90vhujrvcmstl4zr3txmfvw9s5c2epq
-      id: 0
-      sudo_payload: <serialized msg of MessageSudoCallback type>
-    - address: neutron14hj2tavq8fpesdwxxcu44rty3hh90vhujrvcmstl4zr3txmfvw9s5c2epq
-      id: 1
-      sudo_payload: <serialized msg of MessageSudoCallback type>
-  pagination:
-    next_key: null
-    total: "2"
+dispatch: submessages: Generic error: Integrations test mock submsg error: execute wasm contract failed
   ```
 
 </details>

docs/neutron/modules/contract-manager/overview.md

@@ -15,13 +15,29 @@ But this in turn exposes the problem of informing the owner of the contract that
 
 To ensure that the state of the contract is consistent, the call to the sudo handler takes place in a temporary state (using `CacheContext`), which is written to the active state if the call succeeds.
 
+## SudoLimitWrapper
+
+[SudoLimitWrapper](https://github.com/neutron-org/neutron/blob/v2.0.0/x/contractmanager/ibc_middleware.go#L14) is a middleware wrapper with interface
+
+```go
+type WasmKeeper interface {
+    HasContractInfo(ctx sdk.Context, contractAddress sdk.AccAddress) bool
+    Sudo(ctx sdk.Context, contractAddress sdk.AccAddress, msg []byte) ([]byte, error)
+}
+```
+
+It performs two important functions:
+
+1. Make sure that the sudo contract call does not use more gas than allowed by a chain [parameter](#gas-limitation). If the gas limit is exceeded and an out of gas panic occurs inside the sudo call, the wrapper intercepts the panic (only the out of gas panic is intercepted) and converts it to an error.
+2. Capture an error from the sudo handler of the interchaintxs module, either directly initiated by the contract or an error that was received from an out of gas panic and write the error with the data to [Failures](#failures-details) for further processing
+
 ### Gas limitation
 
 To make sure there are no exploits with infinite recursion of IBC messages which call other IBC messages in sudo handler we use constant gas `LIMIT` to spend. The LIMIT is small, so you can't place extensive work in Sudo handlers. As a workaround, in such cases you can use Sudo handlers to simply store required payload in contract's state, and use Execute messages to handle results separately.
 
 If your contract exceeds this constant `LIMIT`, it will terminate sudo handler call and save a `Failure` with full call info. You can [resubmit failure](#resubmitfailure) from this contract.
 
-The `LIMIT` is defined by `SudoCallGasLimit` module's parameter.
+The `LIMIT` is defined by [`SudoCallGasLimit`](https://github.com/neutron-org/neutron/blob/v2.0.2/x/contractmanager/types/params.pb.go#L29) module's parameter.
 
 We provide an ability to resubmit bindings through the contract that initiated the IBC transaction.
 
@@ -67,6 +83,21 @@ Response:
 type FailuresResponse struct {
 	Failures []contractmanagertypes.Failure `json:"failures"`
 }
+
+type Failure struct {
+	// Address of the failed contract
+	Address string `protobuf:"bytes,1,opt,name=address,proto3" json:"address,omitempty"`
+	// Id of the failure under specific address
+	Id uint64 `protobuf:"varint,2,opt,name=id,proto3" json:"id,omitempty"`
+	// Serialized MessageSudoCallback with Packet and Ack(if exists)
+	SudoPayload []byte `protobuf:"bytes,3,opt,name=sudo_payload,json=sudoPayload,proto3" json:"sudo_payload,omitempty"`
+	// Redacted error response of the sudo call. Full error is emitted as an event
+	Error string `protobuf:"bytes,4,opt,name=error,proto3" json:"error,omitempty"`
+}
 ```
 
 You're encouraged to use our neutron-sdk implementation in contracts - [request](https://github.com/neutron-org/neutron-sdk/blob/feat/contract-manager-resubmit/packages/neutron-sdk/src/bindings/query.rs#L61) and [response](https://github.com/neutron-org/neutron-sdk/blob/feat/contract-manager-resubmit/packages/neutron-sdk/src/bindings/query.rs#L119)
+
+## Failures details
+
+In the case of a contract-initiated error, the error is stored in [Failure](state.md). Since raw errors are not part of the consensus and cannot guarantee determinism when saved to the state, errors saved in `Failure` are redacted to `codespace + code_id` using the [RedactError](https://github.com/neutron-org/neutron/blob/v2.0.2/x/contractmanager/keeper/failure.go#L109) function. At the same time, we emit the full text of the error into transaction events that do not affect the consensus and if you need to get detailed information about the error returned by the contract, you need to find the transaction in which the redacted error occurred and see the events in which we emit the error. To simplify this procedure we added a special [cli query](./client.md#failure-details), the error can be found by the `Failure.Address` and the `Failure.Id`.

docs/neutron/modules/contract-manager/state.md

@@ -1,4 +1,9 @@
 # State
 
-The ContractManager module stores one [Failure](https://github.com/neutron-org/neutron/blob/v1.0.4/proto/contractmanager/genesis.proto#L12) per contract address and record id.
-`Failure` contains all the necessary info to store data about ACK sudo handler call failure. `address` contains contract address and it is used in conjunction with  `id` field to create index of the record in the `KVStore`. `ack_id` is used to identify ACK request that was failed, `ack_type` can take two values : `ack`, and `timeout`.
+The ContractManager module stores [Failure](https://github.com/neutron-org/neutron/blob/v2.0.0/proto/neutron/contractmanager/failure.proto#L11) under `contract address` and `record id` key.
+`Failure` contains all the necessary info about ACK sudo handler call failure.
+
+- `address` contains contract address and it is used in conjunction with  
+- `id` field to create index of the record in the `KVStore`.
+- `SudoPayload` - serialized MessageSudoCallback with Packet and Ack(if exists)
+- `Error` - redacted error response of the sudo call. Full error is emitted as an event

docs/neutron/modules/interchain-txs/overview.md

@@ -25,7 +25,7 @@ complete list of IBC events for each module message in the [messages](./messages
 
 ## Sudo errors handling
 
-Interchaintxs module configured the following way, all the errors from a sudo handler are being suppressed by [contract manager middleware](/neutron/modules/contract-manager/overview#concepts), sudo handler is limited with [LIMIT](/neutron/modules/contract-manager/overview#gas-limitation) amount of gas
+Interchaintxs module configured the following way, all the errors from a sudo handler are being suppressed by [contract manager middleware](/neutron/modules/contract-manager/overview#sudolimitwrapper), sudo handler is limited with [LIMIT](/neutron/modules/contract-manager/overview#gas-limitation) amount of gas
 
 ## Importing interchaintxs module
 
@@ -48,6 +48,74 @@ If you use interchaintxs module in your application and if your Sudo handler fai
 > execute [RegisterInterchainAccount message]( /neutron/modules/interchain-txs/messages#msgregisterinterchainaccount) again to
 > recover access to your interchain account.
 > **Note** Keep in mind, new channel is created
+>
+
+## Sudo Handlers
+
+The interchaintxs module in neutrond configured the following way.
+
+Wasmd Sudo handler wrapped with [SudoLimitWrapper](../contract-manager/overview.md#sudolimitwrapper)
+
+And acknoledgement packet follows the way
+`OnAcknowledgementPacket` --> [SudoLimitWrapper](../contract-manager/overview.md#sudolimitwrapper) --> `Wasmd Sudo handler`
+
+Using `SudoLimitWrapper` has two purposes:
+
+1. Suppress the sudo handler error itself, and mark the ibc acknowledgement packet as received and processed. Other way, the error makes relayer send an acknowledgement again and again. Information about an unsuccessfully processed ack is stored in [state](../contract-manager/state.md).
+2. Limit the amount of gas available for sudo handler execution. Out of gas panic will later be captured by `SudoLimitWrapper` and converted into an error.
+
+## Failed interchain txs
+
+Not every interchaintx executes succesfully on a remote network. Some of them fail to execute with errors and then you get ibc acknowledgement with `Error` type. The error is passed into the caller contract via sudo call with SudoMsg::Error [variant](../../../tutorials/cosmwasm_ica.md#ibc-events)
+
+Unfortunately, to avoid the nondeterminism associated with error text generation, the error text is severely truncated by [redact down](https://github.com/cosmos/ibc-go/blob/v7.3.1/modules/apps/27-interchain-accounts/host/ibc_module.go#L115) to the error code without any additional details, before converting into AcknowledgementError.
+
+Find the error text is possible if host chain includes ibc-go v7.2.3+, v7.3.2+, v8.0.1+, v8.1+ which include patch [5541](https://github.com/cosmos/ibc-go/pull/5541)
+`<binary> q interchain-accounts host packet-events <channel-id> <seq-id>`
+
+Where:
+
+- `binary` is a binary on the chain you are working with (the remote chain)
+- `seq-id` - sequence ID of the IBC message sent to the remote chain. The seq-id is returned to the contract in the [SubmitTx](./messages.md#response) response
+- `channel-id` is the ID of the ICA's channel on the remote chain's side. You should know it from registration [procedure](../../../tutorials/cosmwasm-ica.md#2-register-an-interchain-account) via `SudoMsg::OpenAck` from `counterparty_channel_id` field. If you missed it you can always get counterparty channel-id with CLI command `neutrond q ibc channel end <src-port> <src-channel-id>`
+- `src-channel-id` is the channel you intechain account associated with.
+- `src-port` is the port you interchain account is associated with.
+You should know both `src-channel-id` and `src-port` from registration [procedure](../../../tutorials/cosmwasm-ica.md#2-register-an-interchain-account). Also `src-port` is `icacontroller-<contract_address>.<ica_id>` where `ica_id` defined by you during ica registration.
+
+Output example (filtered events):
+
+```json
+{
+"type": "ibccallbackerror-ics27_packet",
+"attributes": [
+    {
+    "key": "ibccallbackerror-module",
+    "value": "interchainaccounts",
+    "index": true
+    },
+    {
+    "key": "ibccallbackerror-host_channel_id",
+    "value": "channel-2",
+    "index": true
+    },
+    {
+    "key": "ibccallbackerror-success",
+    "value": "false",
+    "index": true
+    },
+    {
+    "key": "ibccallbackerror-error",
+    "value": "invalid validator address: decoding bech32 failed: invalid separator index -1: invalid address",
+    "index": true
+    }
+]
+}
+```
+
+On earlier versions of ibc-go it's barely possible to get full text error due to [patch](https://github.com/cosmos/ibc-go/commit/fdbb508c1ca68811206d7175fb9e202c1611a43e).
+
+In the IBC error acknowledgement you get ABCI error and a code, e.g. `codespace: wasm, code: 5`
+where codespace usually is `ModuleName`, and `code` is uniq code for the module. `codespace` and `code` pair uniq for the whole app. You can find the error description in source code. Usualy all the error of the module are [placed](https://github.com/CosmWasm/wasmd/blob/5f444cd9d393513e534cbfa9a0e938295c4e84e1/x/wasm/types/errors.go#L25) in `x/<module>/types/errors.go` where `module` is the module where the error was thrown
 
 ## Relaying