diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 6173927ac3..4e1f5c4e46 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -255,17 +255,19 @@ jobs: env: # This is to skip keygen step RADIXDLT_NODE_KEY: AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAY= - run: ./gradlew :core:run --info & + run: | + echo "db.historical_substate_values.enable=true" >> core/default.config + ./gradlew :core:run --info & - name: Wait for 2 minutes run: sleep 2m - name: Install mesh-cli run: curl -sSfL https://raw.githubusercontent.com/coinbase/mesh-cli/master/scripts/install.sh | sh -s - name: Run Data API tests - run: ./bin/rosetta-cli check:data --configuration-file core-rust/mesh-api-server/mesh-cli-configs/default.json + run: ./bin/rosetta-cli --configuration-file core-rust/mesh-api-server/mesh-cli-configs/localnet.json check:data - name: Run Construction API tests - run: ./bin/rosetta-cli check:construction --configuration-file core-rust/mesh-api-server/mesh-cli-configs/default.json + run: ./bin/rosetta-cli --configuration-file core-rust/mesh-api-server/mesh-cli-configs/localnet.json check:construction - name: Run Coinbase-spec tests - run: ./bin/rosetta-cli check:spec --configuration-file core-rust/mesh-api-server/mesh-cli-configs/default.json + run: ./bin/rosetta-cli --configuration-file core-rust/mesh-api-server/mesh-cli-configs/localnet.json check:spec cross-xwin: name: Cross compile to Windows runs-on: ubuntu-latest diff --git a/core-rust/mesh-api-server/README.md b/core-rust/mesh-api-server/README.md new file mode 100644 index 0000000000..9eba74387e --- /dev/null +++ b/core-rust/mesh-api-server/README.md @@ -0,0 +1,278 @@ +# Mesh API Implementation for Radix + +## Mesh API information + +- [Mesh API Homepage](https://docs.cdp.coinbase.com/mesh/docs/welcome) +- [Mesh API Specification](https://github.com/coinbase/mesh-specifications) +- [Mesh API CLI Test Tool](https://github.com/coinbase/mesh-cli) + +## Supported Features + +| Feature | Status | +| ---------------------------- | ----------------------------------------------------------------------| +| Data API | Feature-complete, with some quirks | +| - `/network/list` | Complete | +| - `/network/status` | Complete | +| - `/network/options` | Complete | +| - `/block` | Feature-complete (exposes only balance-changing operations) | +| - `/block/transaction` | Feature-complete (exposes only balance-changing operations) | +| - `/account/balance` | Complete (historical balances available if explicitly enabled) | +| - `/mempool` | Complete (transactions are not held for a meaningful duration) | +| - `/mempool/transaction` | Complete (basic operations supported) | +| Construction API | Complete | +| - `/construction/derive` | Complete | +| - `/construction/preprocess` | Complete | +| - `/construction/metadata` | Complete | +| - `/construction/payloads` | Complete (supports Withdraw and Deposit operations only) | +| - `/construction/combine` | Complete | +| - `/construction/parse` | Complete (basic operations supported) | +| - `/construction/hash` | Complete | +| - `/construction/submit` | Complete | + +## Additional Considerations + +### Accounts + +The current implementation has the following constraints: +- **Supports only account components**: Returns block operations or balances exclusively for accounts. Other components (e.g., smart contracts, lockers) are ignored. +- **Supports only Withdraw, Deposit, and FeePayment operations**: Minting and burning are skipped. + +These constraints simplify the implementation without causing Mesh CLI tests to fail. If non-account components must be supported, the following may be required: +- Adding support for Minting and Burning operations. +- Providing explicit support for non-account components in balance fetching (or using `dump_component_state()`, which is resource-intensive). +- Exempting some addresses. + +### Operations + +Currently, a very specific parser is used to extract operations from given instructions (endpoints: `/mempool/transaction` and `/construction/parse`). +It works only with the instructions constructed by Mesh. +`Withdraw` and `Deposit` are the direct result of the instructions being used, while a `FeePayment` is added at commit time. + +Technically, it would be possible to use transaction previews, receipts, and balance change summaries to extract operations. +But we don't do it for following reasons: +- both endpoint methods should work offline +- both endpoint methods should be static (not affected by current state of the network) +- this approach is deemed too resource-heavy + +## Configuration + +### Server settings +There are 3 configuration settings for a node's Mesh API server, which can: +- enable/disable Mesh API server launch (disabled by default), +- override the default port (3337), +- override the default bind address (127.0.0.1). + +#### Node running bare-metal +```plaintext +api.mesh.enabled= +api.mesh.port= +api.mesh.bind_address= +``` +#### Node running in Docker +Set below environmental variables + +```plaintext +RADIXDLT_MESH_API_ENABLED= +RADIXDLT_MESH_API_PORT= +RADIXDLT_MESH_API_BIND_ADDRESS= +``` + +### Enable historical balances for reconciliation tests +In order to proceed with reconciliation tests historical balances shall be enabled. +There are 2 useful settings: +- enable/disable historical substate values (disabled by default), +- adjust the state version history length to keep (60000 by default). + +#### Node running bare-metal +```plaintext +db.historical_substate_values.enable= +state_hash_tree.state_version_history_length= +``` + +#### Node running in Docker +``` +RADIXDLT_DB_HISTORICAL_SUBSTATE_VALUES_ENABLE= +RADIXDLT_STATE_HASH_TREE_STATE_VERSION_HISTORY_LENGTH= +``` + +### Base URL + +```plaintext +http://:/mesh +``` + +**Example**: Fetching account balance +```plaintext +http://localhost:3337/mesh/account/balance +``` + +## Testing + +### Mesh CLI + +#### Steps: +1. [Terminal 1] Download the `rosetta-cli` prebuilt binary: + ```bash + curl -sSfL https://raw.githubusercontent.com/coinbase/mesh-cli/master/scripts/install.sh | sh -s + + alias mesh-cli='./bin/rosetta-cli --configuration-file /core-rust/mesh-api-server/mesh-cli-configs/localnet.json' + ``` + **Note:** As of November 2024, there are issues with the prebuilt MacOS binary. Use the workaround below: + ```bash + git clone git@github.com:coinbase/mesh-cli + cd mesh-cli + git checkout bbbd759 + alias mesh-cli='go run main.go --configuration-file /core-rust/mesh-api-server/mesh-cli-configs/localnet.json' + ``` + +2. [Terminal 2] Launch the node: + + - Node running bare-metal + - Change working directory to the root of the `babylon-node` repository + - Enable Mesh API server in `core/default.config` + ```plaintext + api.mesh.enabled=true + ``` + - Optionally setup Mesh port and bind address + ``` + api.mesh.port=3337 + api.mesh.bind_address= + ``` + - For reconciliation tests enable historical balances and optionally set state history length + ``` + db.historical_substate_values.enable=true + state_hash_tree.state_version_history_length=60000 + ``` + + - Start the node + ```bash + RADIXDLT_NODE_KEY=AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAY= ./gradlew :core:run --info + ``` + + - Node running in Docker + + - Manual setup for production or testnet + + Follow these steps to setup a node - [Manual Setup with Docker](https://docs.radixdlt.com/v1/docs/node-setup-docker). + + Before launching a node set Mesh environment variables (mentioned in previous section) in `radix-fullnode-compose.yml` in `core` section: + - Enable Mesh API server: + ```yaml + RADIXDLT_MESH_API_ENABLED: 'true' + ``` + - Optionally setup Mesh port and bind address + ```yaml + RADIXDLT_MESH_API_PORT: 3337 + RADIXDLT_MESH_API_BIND_ADDRESS: '0.0.0.0' + ``` + - For reconciliation tests enable historical balances and optionally set state history length + ```yaml + RADIXDLT_DB_HISTORICAL_SUBSTATE_VALUES_ENABLE: 'true' + RADIXDLT_STATE_HASH_TREE_STATE_VERSION_HISTORY_LENGTH: 60000 + ``` + + - Simple setup for testnet + + Follow these steps to setup a node - [Simple testnet setup](https://github.com/radixdlt/babylon-node/tree/develop/testnet-node). + + Before launching a node set Mesh environment variables (mentioned in previous section) in `radix-node.env`: + - Enable Mesh API server: + ```plaintext + RADIXDLT_MESH_API_ENABLED=true + ``` + - Optionally setup Mesh port and bind address + ```plaintext + RADIXDLT_MESH_API_PORT=3337 + RADIXDLT_MESH_API_BIND_ADDRESS=0.0.0.0 + ``` + - For reconciliation tests enable historical balances and optionally set state history length + ```plaintext + RADIXDLT_DB_HISTORICAL_SUBSTATE_VALUES_ENABLE=true + RADIXDLT_STATE_HASH_TREE_STATE_VERSION_HISTORY_LENGTH=60000 + ``` + +3. [Terminal 1] Run Mesh API tests: + ```bash + mesh-cli check:data + mesh-cli check:construction + mesh-cli check:spec + ``` + +#### Reconciliation Tests + +If whole ledger shall be reconciled for eg. `mainnet` or `stokenet`, then make sure to: +- Set a future `state_version` in the `data.end_conditions.index` field of the `mesh-cli` config file. +- Launch the node with an empty database. +- Start `mesh-cli` as soon as possible to avoid pruning historical balances. + +### Unit Tests + +- **Java:** + ```bash + ./gradlew :core:test --tests '*MeshApiMempoolEndpointsTest*' + ``` +- **Rust:** (To Be Determined) + +## Abstractions + +### NetworkIdentifier + +Fields: +- `blockchain`: "radix" +- `network`: Network variant (e.g., `mainnet`, `stokenet`, `localnet`). +- `sub_network_identifier`: Not set. + +### Block + +Represents a single transaction. + +### BlockIdentifier + +Fields: +- `index`: State version. +- `hash`: Hex-encoded string of 32 bytes composed of: + - `transaction_tree_hash` (bytes[0..12]). + - `receipt_tree_hash` (bytes[0..12]). + - `state_version` (bytes[0..8]). + +### TransactionIdentifier + +- **User transaction:** Bech32-encoded `transaction_intent_hash` (e.g., `txid_tdx_2_1nvm90npmkjcltvpy38nr373pt38ctptgg9y0g3wemhtjxyjmau7s32hd0n`). +- **Non-user transaction:** Bech32-encoded `ledger_transaction_hash` (e.g., `ledgertransaction_tdx_2_1s45u3f6xrh4tf4040aqt9fql3wqlhvwwwfpaz4rsru3pr88f3anstnds7s`). + +### AccountIdentifier + +Fields: +- `address`: Bech32-encoded Global Entity Address. +- `sub_account`: Not set. +- `metadata`: Not set. + +### Currency + +Fields: +- `symbol`: Bech32-encoded Resource Address. +- `decimals`: Resource divisibility. +- `metadata`: Not set. + +### Amount + +Fields: +- `value`: Currency amount. +- `currency`: Resource information. +- `metadata`: Not set. + +### Operation + +Fields: +- `operation_identifier`: Index of the operation within a transaction. +- `related_operations`: Not set. +- `type`: + - `Withdraw`: Withdraw assets from an account (always success, failed operations are filtered out). + - `Deposit`: Deposit assets to an account (always success, failed operations are filtered out). + - `FeePayment`: Withdraw assets to cover transaction fees (always success, even if the transaction fails). +- `status`: Operation status. +- `account`: Account transferring the resources. +- `amount`: Amount of currency transferred. +- `coin_change`: Not set. +- `metadata`: Not set. + diff --git a/core-rust/mesh-api-server/mesh-cli-configs/default.json b/core-rust/mesh-api-server/mesh-cli-configs/localnet.json similarity index 94% rename from core-rust/mesh-api-server/mesh-cli-configs/default.json rename to core-rust/mesh-api-server/mesh-cli-configs/localnet.json index 8e460906d7..52abaad09b 100644 --- a/core-rust/mesh-api-server/mesh-cli-configs/default.json +++ b/core-rust/mesh-api-server/mesh-cli-configs/localnet.json @@ -4,7 +4,7 @@ "network": "localnet" }, "online_url": "http://127.0.0.1:3337/mesh", - "data_directory": "test-data", + "data_directory": "test-data/localnet", "http_timeout": 10, "max_retries": 5, "retry_elapsed_time": 0, @@ -40,6 +40,7 @@ "pruning_block_disabled": false, "pruning_balance_disabled": false, "initial_balance_fetch_disabled": false, + "historical_balance_disabled": false, "end_conditions": { "tip": true, "index": 10000 @@ -47,7 +48,7 @@ }, "construction": { "offline_url": "http://localhost:3337/mesh", - "constructor_dsl_file": "workflows.ros", + "constructor_dsl_file": "localnet.ros", "prefunded_accounts": [ { "account_identifier": { diff --git a/core-rust/mesh-api-server/mesh-cli-configs/workflows.ros b/core-rust/mesh-api-server/mesh-cli-configs/localnet.ros similarity index 100% rename from core-rust/mesh-api-server/mesh-cli-configs/workflows.ros rename to core-rust/mesh-api-server/mesh-cli-configs/localnet.ros diff --git a/core-rust/mesh-api-server/mesh-cli-configs/mainnet.json b/core-rust/mesh-api-server/mesh-cli-configs/mainnet.json new file mode 100644 index 0000000000..30e80e0a4c --- /dev/null +++ b/core-rust/mesh-api-server/mesh-cli-configs/mainnet.json @@ -0,0 +1,59 @@ +{ + "network": { + "blockchain": "radix", + "network": "mainnet" + }, + "online_url": "http://127.0.0.1:3337/mesh", + "data_directory": "test-data/mainnet", + "http_timeout": 10, + "max_retries": 5, + "retry_elapsed_time": 0, + "max_online_connections": 120, + "max_sync_concurrency": 64, + "tip_delay": 300, + "max_reorg_depth": 100, + "log_configuration": false, + "compression_disabled": false, + "l0_in_memory_enabled": false, + "all_in_memory_enabled": false, + "error_stack_trace_disabled": false, + "coin_supported": false, + "data": { + "active_reconciliation_concurrency": 16, + "inactive_reconciliation_concurrency": 4, + "inactive_reconciliation_frequency": 250, + "log_blocks": false, + "log_transactions": false, + "log_balance_changes": false, + "log_reconciliations": false, + "ignore_reconciliation_error": false, + "exempt_accounts": "", + "bootstrap_balances": "", + "interesting_accounts": "", + "reconciliation_disabled": false, + "reconciliation_drain_disabled": false, + "inactive_discrepancy_search_disabled": false, + "balance_tracking_disabled": false, + "coin_tracking_disabled": false, + "status_port": 9090, + "results_output_file": "", + "pruning_block_disabled": false, + "pruning_balance_disabled": false, + "initial_balance_fetch_disabled": false, + "historical_balance_disabled": false, + "end_conditions": { + "index": 104281038 + } + }, + "construction": { + "offline_url": "http://localhost:3337/mesh", + "constructor_dsl_file": "mainnet.ros", + "stale_depth": 1000, + "broadcast_limit": 1000, + "end_conditions": { + "radix_workflow": 1 + } + }, + "perf": null, + "sign": null +} diff --git a/core-rust/mesh-api-server/mesh-cli-configs/mainnet.ros b/core-rust/mesh-api-server/mesh-cli-configs/mainnet.ros new file mode 100644 index 0000000000..67d2ca2ee7 --- /dev/null +++ b/core-rust/mesh-api-server/mesh-cli-configs/mainnet.ros @@ -0,0 +1,64 @@ +radix_workflow(1){ + create_account{ + network = {"network":"mainnet", "blockchain":"radix"}; + key = generate_key({"curve_type":"secp256k1"}); + recipient = derive({ + "network_identifier": {{network}}, + "public_key": {{key.public_key}} + }); + save_account({ + "account_identifier": {{recipient.account_identifier}}, + "keypair": {{key}} + }); + print_message("recipient:"); + print_message({{recipient}}); + }, + transfer{ + currency = {"symbol":"resource_rdx1tknxxxxxxxxxradxrdxxxxxxxxx009923554798xxxxxxxxxradxrd", "decimals":18}; + min_balance = "1111000000000000000000"; + + sender = find_balance({ + "minimum_balance":{ + "value": {{min_balance}}, + "currency": {{currency}} + } + }); + print_message("sender:"); + print_message({{sender}}); + + fee_amount = "22000000000000000000"; + // Negative values are badly interpreted by mesh-cli DSL parser, + // which might lead to errors such as: + // CONSTRUCTION FILE PARSING FAILED! + // Message: failed to parse action: the number of missing variables [deposit_amount] > 0: variable undefined + // Therefore workaround use below approach to specify negative value. + withdraw_amount = "0" - "33000000000000000000"; + deposit_amount = "33000000000000000000"; + + transfer.confirmation_depth = "1"; + transfer.network = {{network}}; + transfer.operations = [ + { + "operation_identifier":{"index":0}, + "type":"Withdraw", + "account":{{sender.account_identifier}}, + "amount":{ + "value":{{withdraw_amount}}, + "currency":{{currency}} + } + }, + { + "operation_identifier":{"index":1}, + "type":"Deposit", + "account":{{recipient.account_identifier}}, + "amount":{ + "value":{{deposit_amount}}, + "currency":{{currency}} + } + } + ]; + + print_message("transfer:"); + print_message({{transfer}}); + } +} diff --git a/core-rust/mesh-api-server/mesh-cli-configs/stokenet.json b/core-rust/mesh-api-server/mesh-cli-configs/stokenet.json new file mode 100644 index 0000000000..1193cb1eec --- /dev/null +++ b/core-rust/mesh-api-server/mesh-cli-configs/stokenet.json @@ -0,0 +1,59 @@ +{ + "network": { + "blockchain": "radix", + "network": "stokenet" + }, + "online_url": "http://127.0.0.1:3337/mesh", + "data_directory": "test-data/stokenet", + "http_timeout": 10, + "max_retries": 5, + "retry_elapsed_time": 0, + "max_online_connections": 120, + "max_sync_concurrency": 64, + "tip_delay": 300, + "max_reorg_depth": 100, + "log_configuration": false, + "compression_disabled": false, + "l0_in_memory_enabled": false, + "all_in_memory_enabled": false, + "error_stack_trace_disabled": false, + "coin_supported": false, + "data": { + "active_reconciliation_concurrency": 16, + "inactive_reconciliation_concurrency": 4, + "inactive_reconciliation_frequency": 250, + "log_blocks": false, + "log_transactions": false, + "log_balance_changes": false, + "log_reconciliations": false, + "ignore_reconciliation_error": false, + "exempt_accounts": "", + "bootstrap_balances": "", + "interesting_accounts": "", + "reconciliation_disabled": false, + "reconciliation_drain_disabled": false, + "inactive_discrepancy_search_disabled": false, + "balance_tracking_disabled": false, + "coin_tracking_disabled": false, + "status_port": 9090, + "results_output_file": "", + "pruning_block_disabled": false, + "pruning_balance_disabled": false, + "initial_balance_fetch_disabled": false, + "historical_balance_disabled": false, + "end_conditions": { + "index": 104281038 + } + }, + "construction": { + "offline_url": "http://localhost:3337/mesh", + "constructor_dsl_file": "stokenet.ros", + "stale_depth": 1000, + "broadcast_limit": 1000, + "end_conditions": { + "radix_workflow": 1 + } + }, + "perf": null, + "sign": null +} diff --git a/core-rust/mesh-api-server/mesh-cli-configs/stokenet.ros b/core-rust/mesh-api-server/mesh-cli-configs/stokenet.ros new file mode 100644 index 0000000000..0e5e2c978e --- /dev/null +++ b/core-rust/mesh-api-server/mesh-cli-configs/stokenet.ros @@ -0,0 +1,64 @@ +radix_workflow(1){ + create_account{ + network = {"network":"stokenet", "blockchain":"radix"}; + key = generate_key({"curve_type":"secp256k1"}); + recipient = derive({ + "network_identifier": {{network}}, + "public_key": {{key.public_key}} + }); + save_account({ + "account_identifier": {{recipient.account_identifier}}, + "keypair": {{key}} + }); + print_message("recipient:"); + print_message({{recipient}}); + }, + transfer{ + currency = {"symbol":"resource_tdx_2_1tknxxxxxxxxxradxrdxxxxxxxxx009923554798xxxxxxxxxtfd2jc", "decimals":18}; + min_balance = "1111000000000000000000"; + + sender = find_balance({ + "minimum_balance":{ + "value": {{min_balance}}, + "currency": {{currency}} + } + }); + print_message("sender:"); + print_message({{sender}}); + + fee_amount = "22000000000000000000"; + // Negative values are badly interpreted by mesh-cli DSL parser, + // which might lead to errors such as: + // CONSTRUCTION FILE PARSING FAILED! + // Message: failed to parse action: the number of missing variables [deposit_amount] > 0: variable undefined + // Therefore workaround use below approach to specify negative value. + withdraw_amount = "0" - "33000000000000000000"; + deposit_amount = "33000000000000000000"; + + transfer.confirmation_depth = "1"; + transfer.network = {{network}}; + transfer.operations = [ + { + "operation_identifier":{"index":0}, + "type":"Withdraw", + "account":{{sender.account_identifier}}, + "amount":{ + "value":{{withdraw_amount}}, + "currency":{{currency}} + } + }, + { + "operation_identifier":{"index":1}, + "type":"Deposit", + "account":{{recipient.account_identifier}}, + "amount":{ + "value":{{deposit_amount}}, + "currency":{{currency}} + } + } + ]; + + print_message("transfer:"); + print_message({{transfer}}); + } +} diff --git a/core-rust/mesh-api-server/src/mesh_api/conversions/addressing.rs b/core-rust/mesh-api-server/src/mesh_api/conversions/addressing.rs index be650db91b..5cfef52e9c 100644 --- a/core-rust/mesh-api-server/src/mesh_api/conversions/addressing.rs +++ b/core-rust/mesh-api-server/src/mesh_api/conversions/addressing.rs @@ -18,7 +18,7 @@ pub fn extract_component_address( pub(crate) fn extract_resource_address_from_currency( extraction_context: &ExtractionContext, - database: &StateManagerDatabase, + database: &impl SubstateDatabase, currency: &models::Currency, ) -> Result { // currency.symbol field keeps bech32-encoded resource address @@ -30,7 +30,6 @@ pub(crate) fn extract_resource_address_from_currency( message: format!("currency {} is not fungible type", currency.symbol), }); } - let divisibility: FungibleResourceManagerDivisibilityFieldSubstate = read_optional_main_field_substate( database, @@ -77,7 +76,6 @@ pub fn to_api_account_identifier_from_global_address( let node_id: &NodeId = address.as_ref(); let address = to_api_entity_address(mapping_context, node_id)?; - // TODO:MESH remove account filtering if !node_id.is_global_account() { return Err(MappingError::InvalidAccount { message: format!("address {} is not an account", address), diff --git a/core-rust/mesh-api-server/src/mesh_api/conversions/block.rs b/core-rust/mesh-api-server/src/mesh_api/conversions/block.rs index b729aee398..127992bde9 100644 --- a/core-rust/mesh-api-server/src/mesh_api/conversions/block.rs +++ b/core-rust/mesh-api-server/src/mesh_api/conversions/block.rs @@ -93,18 +93,11 @@ pub fn extract_state_version_from_mesh_api_block_identifier( } pub fn to_mesh_api_block_identifier_from_state_version( - database: &StateManagerDatabase, state_version: StateVersion, + transaction_tree_hash: &TransactionTreeHash, + receipt_tree_hash: &ReceiptTreeHash, ) -> Result { let index = to_mesh_api_block_index_from_state_version(state_version)?; - let transaction_identifiers = database - .get_committed_transaction_identifiers(state_version) - .ok_or_else(|| MappingError::TransactionNotFound)?; - - let transaction_tree_hash = transaction_identifiers - .resultant_ledger_hashes - .transaction_root; - let receipt_tree_hash = transaction_identifiers.resultant_ledger_hashes.receipt_root; let mut hash_bytes = [0u8; 32]; @@ -119,10 +112,13 @@ pub fn to_mesh_api_block_identifier_from_state_version( } pub fn to_mesh_api_block_identifier_from_ledger_header( - database: &StateManagerDatabase, ledger_header: &LedgerStateSummary, ) -> Result { - to_mesh_api_block_identifier_from_state_version(database, ledger_header.state_version) + to_mesh_api_block_identifier_from_state_version( + ledger_header.state_version, + &ledger_header.hashes.transaction_root, + &ledger_header.hashes.receipt_root, + ) } pub fn to_mesh_api_block_index_from_state_version( diff --git a/core-rust/mesh-api-server/src/mesh_api/conversions/currency.rs b/core-rust/mesh-api-server/src/mesh_api/conversions/currency.rs index 5cdccac3aa..92a058798c 100644 --- a/core-rust/mesh-api-server/src/mesh_api/conversions/currency.rs +++ b/core-rust/mesh-api-server/src/mesh_api/conversions/currency.rs @@ -2,7 +2,7 @@ use crate::prelude::*; pub fn to_mesh_api_currency_from_resource_address( mapping_context: &MappingContext, - database: &StateManagerDatabase, + database: &impl SubstateDatabase, resource_address: &ResourceAddress, ) -> Result { let resource_node_id = resource_address.as_node_id(); @@ -14,7 +14,6 @@ pub fn to_mesh_api_currency_from_resource_address( message: format!("resource {} is not fungible type", symbol), }); } - let divisibility: FungibleResourceManagerDivisibilityFieldSubstate = read_optional_main_field_substate( database, diff --git a/core-rust/mesh-api-server/src/mesh_api/conversions/numerics.rs b/core-rust/mesh-api-server/src/mesh_api/conversions/numerics.rs index 122dfb0dfd..ae5b5c6d9d 100644 --- a/core-rust/mesh-api-server/src/mesh_api/conversions/numerics.rs +++ b/core-rust/mesh-api-server/src/mesh_api/conversions/numerics.rs @@ -52,3 +52,66 @@ pub fn to_mesh_api_amount( Ok(models::Amount::new(value.attos().to_string(), currency)) } + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn test_amount_extraction_decimals() { + let extraction_context = ExtractionContext::new(&NetworkDefinition::localnet()); + let mapping_context = MappingContext::new(&NetworkDefinition::localnet()); + + let xrd_str = to_api_resource_address(&mapping_context, &XRD).unwrap(); + + for decimals in 0..18 { + let currency = models::Currency { + symbol: xrd_str.clone(), + decimals, + metadata: None, + }; + + let mesh_api_amount = to_mesh_api_amount(dec!(200), currency).unwrap(); + + assert_eq!( + extract_amount(&extraction_context, &mesh_api_amount).unwrap(), + (XRD, dec!(200)) + ); + } + } + + #[test] + fn test_amount_extraction_amounts() { + let extraction_context = ExtractionContext::new(&NetworkDefinition::localnet()); + let mapping_context = MappingContext::new(&NetworkDefinition::localnet()); + + let xrd_str = to_api_resource_address(&mapping_context, &XRD).unwrap(); + + let currency = models::Currency { + symbol: xrd_str.clone(), + decimals: 1, + metadata: None, + }; + + let amount = dec!(200.1); + let mesh_api_amount = to_mesh_api_amount(amount, currency).unwrap(); + assert_eq!( + extract_amount(&extraction_context, &mesh_api_amount).unwrap(), + (XRD, amount) + ); + + let currency = models::Currency { + symbol: xrd_str.clone(), + decimals: 2, + metadata: None, + }; + + // Surplus decimals are truncated. + // In fact, decimal mismatch should never be observable. + let mesh_api_amount = to_mesh_api_amount(dec!(200.027), currency).unwrap(); + assert_eq!( + extract_amount(&extraction_context, &mesh_api_amount).unwrap(), + (XRD, dec!(200.02)) + ); + } +} diff --git a/core-rust/mesh-api-server/src/mesh_api/conversions/operations.rs b/core-rust/mesh-api-server/src/mesh_api/conversions/operations.rs index 5f65788dd8..4187392e0a 100644 --- a/core-rust/mesh-api-server/src/mesh_api/conversions/operations.rs +++ b/core-rust/mesh-api-server/src/mesh_api/conversions/operations.rs @@ -1,10 +1,16 @@ use crate::engine_prelude::*; use crate::prelude::*; +use radix_engine_interface::blueprints::account::{ + AccountTryDepositBatchOrAbortManifestInput, AccountTryDepositOrAbortManifestInput, + AccountWithdrawManifestInput, +}; +use radix_transactions::manifest::{CallMethod, TakeFromWorktop}; #[derive(Debug, Clone, Copy, EnumIter, Display, EnumString)] pub(crate) enum MeshApiOperationType { Withdraw, Deposit, + FeePayment, } #[derive(Debug, Clone, Copy, EnumIter, Display)] @@ -15,17 +21,6 @@ pub(crate) enum MeshApiOperationStatus { Failure, } -// TODO:MESH This one might be confusing. Failed transaction will still have successful FeePayment -// operation -impl From for MeshApiOperationStatus { - fn from(value: DetailedTransactionOutcome) -> Self { - match value { - DetailedTransactionOutcome::Success(..) => Self::Success, - DetailedTransactionOutcome::Failure(..) => Self::Failure, - } - } -} - impl From for models::OperationStatus { fn from(value: MeshApiOperationStatus) -> Self { let successful = match value { @@ -45,7 +40,6 @@ pub fn to_mesh_api_operation_no_fee( resource_address: &ResourceAddress, amount: Decimal, ) -> Result { - // TODO:MESH what about fee locking, burning, minting? let op_type = if amount.is_positive() { MeshApiOperationType::Deposit } else { @@ -68,6 +62,30 @@ pub fn to_mesh_api_operation_no_fee( }) } +pub fn to_mesh_api_operation_fee_payment( + mapping_context: &MappingContext, + database: &StateManagerDatabase, + index: i64, + account_address: &GlobalAddress, + amount: Decimal, +) -> Result { + let currency = to_mesh_api_currency_from_resource_address(mapping_context, database, &XRD)?; + let account = to_api_account_identifier_from_global_address(mapping_context, account_address)?; + + // see https://docs.cdp.coinbase.com/mesh/docs/models#operation + Ok(models::Operation { + operation_identifier: Box::new(models::OperationIdentifier::new(index)), + related_operations: None, + _type: MeshApiOperationType::FeePayment.to_string(), + // Fee payment is always success, even if transaction failed + status: Some(MeshApiOperationStatus::Success.to_string()), + account: Some(Box::new(account)), + amount: Some(Box::new(to_mesh_api_amount(amount, currency)?)), + coin_change: None, + metadata: None, + }) +} + pub fn to_mesh_api_operations( mapping_context: &MappingContext, database: &StateManagerDatabase, @@ -81,7 +99,6 @@ pub fn to_mesh_api_operations( state_version.number() ), })?; - let status = MeshApiOperationStatus::from(local_execution.outcome); let fee_balance_changes = resolve_global_fee_balance_changes(database, &local_execution.fee_source)?; @@ -97,6 +114,30 @@ pub fn to_mesh_api_operations( let mut output = Vec::with_capacity(fee_payment_computation.relevant_entities.len()); for entity in &fee_payment_computation.relevant_entities { if entity.is_account() { + if let Some(fee_balance_changes) = + fee_payment_computation.fee_balance_changes.get(&entity) + { + for amount in fee_balance_changes + .iter() + .filter_map(|(fee_payment_type, amount)| { + if *fee_payment_type == FeePaymentBalanceChangeType::FeePayment { + Some(amount) + } else { + None + } + }) + { + let operation = to_mesh_api_operation_fee_payment( + mapping_context, + database, + output.len() as i64, + entity, + *amount, + )?; + output.push(operation) + } + } + if let Some(non_fee_balance_changes) = fee_payment_computation.non_fee_balance_changes.get(&entity) { @@ -105,7 +146,8 @@ pub fn to_mesh_api_operations( mapping_context, database, output.len() as i64, - Some(status), + // There are no non-fee balance changes for failed transactions + Some(MeshApiOperationStatus::Success), entity, resource_address, *amount, @@ -140,3 +182,128 @@ fn resolve_global_fee_balance_changes( } Ok(fee_balance_changes) } + +/// This method converts Transaction V1 instructons to operations. +/// The parser is limited to the instructions generated by the Mesh construction API. +pub fn to_mesh_api_operations_from_instructions_v1( + instructions: &[InstructionV1], + mapping_context: &MappingContext, + database: &StateManagerDatabase, +) -> Result, ResponseError> { + let mut operations = Vec::new(); + let mut next_index = 0; + let mut withdraw_input: Option<(ResourceAddress, Decimal)> = None; + + while next_index < instructions.len() { + let mut instruction = &instructions[next_index]; + next_index = next_index + 1; + match instruction { + InstructionV1::CallMethod(CallMethod { + address: DynamicGlobalAddress::Static(global_address), + method_name, + args, + }) => { + let args_bytes = manifest_encode(&args).unwrap(); + match method_name.as_str() { + "lock_fee" => (), + "withdraw" if global_address.is_account() => { + let input = manifest_decode::(&args_bytes) + .map_err(|_| { + ResponseError::from(ApiError::InvalidManifestInstruction) + .with_details("Invalid withdraw instruction") + })?; + let resource_adddress = &match input.resource_address { + ManifestResourceAddress::Static(resource_address) => resource_address, + ManifestResourceAddress::Named(_) => { + return Err(ResponseError::from(ApiError::NamedAddressNotSupported) + .with_details("Named address is not supported")) + } + }; + + operations.push(to_mesh_api_operation_no_fee( + mapping_context, + database, + operations.len() as i64, + None, + global_address, + resource_adddress, + -input.amount.clone(), + )?); + withdraw_input = Some((*resource_adddress, input.amount)); + } + // Below assumes that previous operation was Withdraw and whole withdraw amount + // shall be deposited to the global address + "try_deposit_batch_or_abort" + if global_address.is_account() && withdraw_input.is_some() => + { + let (resource_address, amount) = withdraw_input.unwrap(); + if let Ok(_input) = manifest_decode::< + AccountTryDepositBatchOrAbortManifestInput, + >(&args_bytes) + { + operations.push(to_mesh_api_operation_no_fee( + mapping_context, + database, + operations.len() as i64, + None, + global_address, + &resource_address, + amount, + )?); + } else { + return Err(ResponseError::from(ApiError::InvalidManifestInstruction) + .with_details("Invalid try_deposit_batch_or_abort instruction")); + } + } + _ => { + return Err(ResponseError::from(ApiError::UnrecognizedInstruction) + .with_details(format!("Unrecognized instruction: {:?}", instruction))); + } + } + } + InstructionV1::TakeFromWorktop(TakeFromWorktop { + resource_address, + amount, + }) if next_index < instructions.len() => { + instruction = &instructions[next_index]; + next_index = next_index + 1; + + match instruction { + InstructionV1::CallMethod(CallMethod { + address: DynamicGlobalAddress::Static(global_address), + method_name, + args, + }) if method_name.eq("try_deposit_or_abort") && global_address.is_account() => { + if let Ok(_input) = manifest_decode::( + &manifest_encode(&args).unwrap(), + ) { + operations.push(to_mesh_api_operation_no_fee( + mapping_context, + database, + operations.len() as i64, + None, + global_address, + resource_address, + *amount, + )?); + } else { + return Err(ResponseError::from(ApiError::InvalidManifestInstruction) + .with_details("Invalid try_deposit_or_abort instruction")); + } + } + _ => { + return Err(ResponseError::from(ApiError::UnrecognizedInstruction) + .with_details(format!("Unrecognized instruction: {:?}", instruction))); + } + } + withdraw_input = None; + } + _ => { + return Err(ResponseError::from(ApiError::UnrecognizedInstruction) + .with_details(format!("Unrecognized instruction: {:?}", instruction))); + } + } + } + + Ok(operations) +} diff --git a/core-rust/mesh-api-server/src/mesh_api/errors.rs b/core-rust/mesh-api-server/src/mesh_api/errors.rs index 6f0b54173d..148b7139b8 100644 --- a/core-rust/mesh-api-server/src/mesh_api/errors.rs +++ b/core-rust/mesh-api-server/src/mesh_api/errors.rs @@ -31,8 +31,8 @@ pub(crate) enum ApiError { InvalidNumberOfSignatures, #[strum(serialize = "Invalid transaction")] InvalidTransaction, - #[strum(serialize = "Invalid Withdraw instruction")] - InvalidWithdrawInstruction, + #[strum(serialize = "Invalid manifest instruction")] + InvalidManifestInstruction, #[strum(serialize = "Named address not supported")] NamedAddressNotSupported, #[strum(serialize = "Instruction is not recognized")] @@ -51,6 +51,8 @@ pub(crate) enum ApiError { InvalidBlockIdentifier, #[strum(serialize = "Submit transaction error")] SubmitTransactionError, + #[strum(serialize = "Get state history error")] + GetStateHistoryError, } impl From for ResponseError { diff --git a/core-rust/mesh-api-server/src/mesh_api/handlers/account_balance.rs b/core-rust/mesh-api-server/src/mesh_api/handlers/account_balance.rs index 36753b0e3e..71bc911777 100644 --- a/core-rust/mesh-api-server/src/mesh_api/handlers/account_balance.rs +++ b/core-rust/mesh-api-server/src/mesh_api/handlers/account_balance.rs @@ -16,31 +16,40 @@ pub(crate) async fn handle_account_balance( .map_err(|err| err.into_response_error("account_identifier"))?; let database = state.state_manager.database.snapshot(); - - let header = if request.block_identifier.is_some() { - return Err(ResponseError::from(ApiError::InvalidRequest) - .with_details("Historical balance not supported")); + let state_version = if let Some(block_identifier) = request.block_identifier { + extract_state_version_from_mesh_api_partial_block_identifier( + database.deref(), + &block_identifier, + ) + .map_err(|err| err.into_response_error("block_identifier"))? } else { - read_current_ledger_header(database.deref()) + None }; + let scoped_database = database.scoped_at(state_version).map_err(|err| { + ResponseError::from(ApiError::GetStateHistoryError) + .with_details(format!("Getting state history error: {:?}", err)) + })?; + let balances = match request.currencies { Some(currencies) => { let resources = currencies .into_iter() - .map(|c| { + .filter_map(|c| { + // Filter out resources, which were not possible to extract, + // eg. not found because they were not existing at given state version extract_resource_address_from_currency( &extraction_context, - database.deref(), + &scoped_database, &c, ) + .ok() }) - .collect::, ExtractionError>>() - .map_err(|err| err.into_response_error("currency"))?; + .collect::>(); get_requested_balances( &mapping_context, - database.deref(), + &scoped_database, &component_address, &resources, )? @@ -48,28 +57,29 @@ pub(crate) async fn handle_account_balance( None => { // Check if account is instantiated let type_info: Option = read_optional_substate::( - database.deref(), + &scoped_database, component_address.as_node_id(), TYPE_INFO_FIELD_PARTITION, &TypeInfoField::TypeInfo.into(), ); if type_info.is_some() { - get_all_balances(&mapping_context, database.deref(), &component_address)? + get_all_balances(&mapping_context, &scoped_database, &component_address)? } else { // We expect empty balances vector here, but let the `get_requested_balances()` // deal with this. - get_requested_balances(&mapping_context, database.deref(), &component_address, &[])? + get_requested_balances(&mapping_context, &scoped_database, &component_address, &[])? } } }; + let ledger_state = scoped_database.at_ledger_state(); + // see https://docs.cdp.coinbase.com/mesh/docs/models#accountbalanceresponse for field // definitions Ok(Json(models::AccountBalanceResponse { block_identifier: Box::new(to_mesh_api_block_identifier_from_ledger_header( - database.deref(), - &header.into(), + &ledger_state, )?), balances, metadata: None, @@ -78,9 +88,9 @@ pub(crate) async fn handle_account_balance( // Method `dump_component_state()` might be slow on large accounts, // therefore we use it only when user didn't specify which balances // to get. -fn get_all_balances( +fn get_all_balances<'a>( mapping_context: &MappingContext, - database: &StateManagerDatabase, + database: &impl SubstateDatabase, component_address: &ComponentAddress, ) -> Result, MappingError> { let component_dump = dump_component_state(database, *component_address); @@ -114,44 +124,47 @@ fn get_all_balances( fn get_requested_balances( mapping_context: &MappingContext, - database: &StateManagerDatabase, + database: &impl SubstateDatabase, component_address: &ComponentAddress, resource_addresses: &[ResourceAddress], ) -> Result, ResponseError> { - resource_addresses.into_iter().map(|resource_address| { - let balance = { - let encoded_key = scrypto_encode(resource_address).expect("Impossible Case!"); - let substate = read_optional_collection_substate::( - database, - component_address.as_node_id(), - AccountCollection::ResourceVaultKeyValue.collection_index(), - &SubstateKey::Map(encoded_key), - ); - match substate { - Some(substate) => { - let vault = substate - .into_value() - .ok_or(MappingError::KeyValueStoreEntryUnexpectedlyAbsent)? - .fully_update_and_into_latest_version(); - read_mandatory_main_field_substate::( + resource_addresses + .into_iter() + .map(|resource_address| { + let balance = { + let encoded_key = scrypto_encode(resource_address).expect("Impossible Case!"); + let substate = + read_optional_collection_substate::( database, - vault.0.as_node_id(), - &FungibleVaultField::Balance.into(), - )? - .into_payload() - .fully_update_and_into_latest_version() - .amount() + component_address.as_node_id(), + AccountCollection::ResourceVaultKeyValue.collection_index(), + &SubstateKey::Map(encoded_key), + ); + match substate { + Some(substate) => { + let vault = substate + .into_value() + .ok_or(MappingError::KeyValueStoreEntryUnexpectedlyAbsent)? + .fully_update_and_into_latest_version(); + read_mandatory_main_field_substate::( + database, + vault.0.as_node_id(), + &FungibleVaultField::Balance.into(), + )? + .into_payload() + .fully_update_and_into_latest_version() + .amount() + } + _ => Decimal::ZERO, } - _ => Decimal::ZERO, - } - }; + }; - let currency = to_mesh_api_currency_from_resource_address( - &mapping_context, - database, - &resource_address, - )?; - Ok(to_mesh_api_amount(balance, currency)?) - }) - .collect::, ResponseError>>() + let currency = to_mesh_api_currency_from_resource_address( + &mapping_context, + database, + &resource_address, + )?; + Ok(to_mesh_api_amount(balance, currency)?) + }) + .collect::, ResponseError>>() } diff --git a/core-rust/mesh-api-server/src/mesh_api/handlers/block.rs b/core-rust/mesh-api-server/src/mesh_api/handlers/block.rs index 527e5175d3..0f518b1e48 100644 --- a/core-rust/mesh-api-server/src/mesh_api/handlers/block.rs +++ b/core-rust/mesh-api-server/src/mesh_api/handlers/block.rs @@ -31,6 +31,14 @@ pub(crate) async fn handle_block( state_version.number() )) })?; + let previous_transaction_identifiers = database + .get_committed_transaction_identifiers(previous_state_version) + .ok_or_else(|| { + ResponseError::from(ApiError::TransactionNotFound).with_details(format!( + "Failed fetching transaction identifiers for state version {}", + previous_state_version.number() + )) + })?; let operations = to_mesh_api_operations(&mapping_context, database.deref(), state_version)?; @@ -48,12 +56,20 @@ pub(crate) async fn handle_block( // see https://docs.cdp.coinbase.com/mesh/docs/models#block let block = models::Block { block_identifier: Box::new(to_mesh_api_block_identifier_from_state_version( - database.deref(), state_version, + &transaction_identifiers + .resultant_ledger_hashes + .transaction_root, + &transaction_identifiers.resultant_ledger_hashes.receipt_root, )?), parent_block_identifier: Box::new(to_mesh_api_block_identifier_from_state_version( - database.deref(), previous_state_version, + &previous_transaction_identifiers + .resultant_ledger_hashes + .transaction_root, + &previous_transaction_identifiers + .resultant_ledger_hashes + .receipt_root, )?), timestamp: transaction_identifiers.proposer_timestamp_ms, transactions: vec![transaction], diff --git a/core-rust/mesh-api-server/src/mesh_api/handlers/construction_combine.rs b/core-rust/mesh-api-server/src/mesh_api/handlers/construction_combine.rs index 9968970afe..2352f1887f 100644 --- a/core-rust/mesh-api-server/src/mesh_api/handlers/construction_combine.rs +++ b/core-rust/mesh-api-server/src/mesh_api/handlers/construction_combine.rs @@ -18,15 +18,20 @@ pub(crate) async fn handle_construction_combine( ); }; - let intent = RawTransactionIntent::from_hex(&request.unsigned_transaction) - .ok() - .and_then(|x| IntentV1::from_raw(&x).ok()) - .ok_or( - ResponseError::from(ApiError::InvalidTransaction).with_details(format!( - "Invalid unsigned transaction: {}", - &request.unsigned_transaction - )), - )?; + let raw = RawTransactionIntent::from_hex(&request.unsigned_transaction).map_err(|_| { + ResponseError::from(ApiError::InvalidTransaction).with_details(format!( + "Invalid transaction hex: {}", + &request.unsigned_transaction + )) + })?; + + let intent = IntentV1::from_raw(&raw).map_err(|err| { + ResponseError::from(ApiError::InvalidTransaction).with_details(format!( + "Failed to create transaction intent v1 from raw: {:?}", + err + )) + })?; + let tx = NotarizedTransactionV1 { signed_intent: SignedIntentV1 { intent, diff --git a/core-rust/mesh-api-server/src/mesh_api/handlers/construction_hash.rs b/core-rust/mesh-api-server/src/mesh_api/handlers/construction_hash.rs index 894470e1e1..8ae200a3b5 100644 --- a/core-rust/mesh-api-server/src/mesh_api/handlers/construction_hash.rs +++ b/core-rust/mesh-api-server/src/mesh_api/handlers/construction_hash.rs @@ -7,16 +7,18 @@ pub(crate) async fn handle_construction_hash( assert_matching_network(&request.network_identifier, &state.network)?; let intent_hash = RawNotarizedTransaction::from_hex(&request.signed_transaction) - .ok() - .and_then(|raw| raw.prepare(PreparationSettings::latest_ref()).ok()) - .and_then(|tx| Some(tx.hashes())) - .ok_or( + .map_err(|_| { ResponseError::from(ApiError::InvalidTransaction).with_details(format!( - "Invalid transaction: {}", - request.signed_transaction - )), - )? - .transaction_intent_hash; + "Invalid transaction hex: {}", + &request.signed_transaction + )) + })? + .prepare(PreparationSettings::latest_ref()) + .map_err(|err| { + ResponseError::from(ApiError::InvalidTransaction) + .with_details(format!("Failed to prepare user transaction: {:?}", err)) + })? + .transaction_intent_hash(); let transaction_identifier = to_mesh_api_transaction_identifier_from_hash( to_api_transaction_hash_bech32m(&MappingContext::new(&state.network), &intent_hash)?, diff --git a/core-rust/mesh-api-server/src/mesh_api/handlers/construction_parse.rs b/core-rust/mesh-api-server/src/mesh_api/handlers/construction_parse.rs index 2d5ca80774..12cfa522b6 100644 --- a/core-rust/mesh-api-server/src/mesh_api/handlers/construction_parse.rs +++ b/core-rust/mesh-api-server/src/mesh_api/handlers/construction_parse.rs @@ -1,8 +1,4 @@ use crate::prelude::*; -use radix_engine_interface::blueprints::account::{ - AccountTryDepositOrAbortManifestInput, AccountWithdrawManifestInput, -}; -use radix_transactions::manifest::{CallMethod, TakeFromWorktop}; use radix_transactions::validation::TransactionValidator; // This method only accepts transactions constructed with the Mesh API, @@ -21,19 +17,25 @@ pub(crate) async fn handle_construction_parse( })?; let (instructions, signers) = if request.signed { - let transaction = - NotarizedTransactionV1::from_raw(&RawNotarizedTransaction::from_vec(transaction_bytes)) - .map_err(|_| { - ResponseError::from(ApiError::InvalidTransaction) - .with_details(format!("Invalid transaction: {}", &request.transaction)) - })?; + let transaction = match manifest_decode::(&transaction_bytes) { + Ok(AnyTransaction::NotarizedTransactionV1(transaction)) => transaction, + Ok(_) => { + return Err(ResponseError::from(ApiError::InvalidTransaction) + .with_details("Only V1 notarized transactions are supported in the Mesh API parse endpoint")); + } + Err(_) => { + return Err(ResponseError::from(ApiError::InvalidTransaction) + .with_details(format!("Invalid transaction: {}", &request.transaction))) + } + }; + let validated = transaction .prepare_and_validate(&TransactionValidator::new_with_latest_config( &state.network, )) .map_err(|e| { ResponseError::from(ApiError::InvalidTransaction) - .with_details(format!("Invalid transaction: error = {:?}", e)) + .with_details(format!("Transaction validation error: {:?}", e)) })?; let instructions = transaction.signed_intent.intent.instructions.0; @@ -53,7 +55,11 @@ pub(crate) async fn handle_construction_parse( let mapping_context = MappingContext::new(&state.network); let database = state.state_manager.database.snapshot(); - let operations = parse_instructions(&instructions, &mapping_context, database.deref())?; + let operations = to_mesh_api_operations_from_instructions_v1( + &instructions, + &mapping_context, + database.deref(), + )?; // See https://docs.cdp.coinbase.com/mesh/docs/models#constructionparseresponse for field // definitions @@ -71,97 +77,3 @@ pub(crate) async fn handle_construction_parse( metadata: None, })) } - -pub fn parse_instructions( - instructions: &[InstructionV1], - mapping_context: &MappingContext, - database: &StateManagerDatabase, -) -> Result, ResponseError> { - let mut operations = Vec::new(); - let mut next_index = 0; - while next_index < instructions.len() { - let mut instruction = &instructions[next_index]; - next_index = next_index + 1; - match instruction { - InstructionV1::CallMethod(CallMethod { - address: DynamicGlobalAddress::Static(global_address), - method_name, - args, - }) if global_address.is_account() => { - let args_bytes = manifest_encode(&args).unwrap(); - match method_name.as_str() { - "lock_fee" => (), - "withdraw" => { - let input = manifest_decode::(&args_bytes) - .map_err(|_| { - ResponseError::from(ApiError::InvalidWithdrawInstruction) - .with_details("Invalid withdraw instruction") - })?; - operations.push(to_mesh_api_operation_no_fee( - mapping_context, - database, - operations.len() as i64, - None, - global_address, - &match input.resource_address { - ManifestResourceAddress::Static(resource_address) => { - resource_address - } - ManifestResourceAddress::Named(_) => { - return Err(ResponseError::from( - ApiError::NamedAddressNotSupported, - ) - .with_details("Named address is not supported")) - } - }, - -input.amount.clone(), - )?); - } - _ => { - return Err(ResponseError::from(ApiError::UnrecognizedInstruction) - .with_details(format!("Unrecognized instruction: {:?}", instruction))); - } - } - } - InstructionV1::TakeFromWorktop(TakeFromWorktop { - resource_address, - amount, - }) if next_index < instructions.len() => { - instruction = &instructions[next_index]; - next_index = next_index + 1; - - match instruction { - InstructionV1::CallMethod(CallMethod { - address: DynamicGlobalAddress::Static(global_address), - method_name, - args, - }) if method_name.eq("try_deposit_or_abort") && global_address.is_account() => { - if let Ok(_input) = manifest_decode::( - &manifest_encode(&args).unwrap(), - ) { - operations.push(to_mesh_api_operation_no_fee( - mapping_context, - database, - operations.len() as i64, - None, - global_address, - resource_address, - *amount, - )?); - } - } - _ => { - return Err(ResponseError::from(ApiError::UnrecognizedInstruction) - .with_details(format!("Unrecognized instruction: {:?}", instruction))); - } - } - } - _ => { - return Err(ResponseError::from(ApiError::UnrecognizedInstruction) - .with_details(format!("Unrecognized instruction: {:?}", instruction))); - } - } - } - - Ok(operations) -} diff --git a/core-rust/mesh-api-server/src/mesh_api/handlers/construction_payloads.rs b/core-rust/mesh-api-server/src/mesh_api/handlers/construction_payloads.rs index ef9d23253b..4544707f74 100644 --- a/core-rust/mesh-api-server/src/mesh_api/handlers/construction_payloads.rs +++ b/core-rust/mesh-api-server/src/mesh_api/handlers/construction_payloads.rs @@ -80,6 +80,8 @@ pub(crate) async fn handle_construction_payloads( builder = builder.take_from_worktop(address, quantity, &bucket); builder = builder.try_deposit_or_abort(account, None, bucket); } + // At the moment of construction we cannot determine the fee amount - skip it + MeshApiOperationType::FeePayment => (), } } let manifest = builder.build(); diff --git a/core-rust/mesh-api-server/src/mesh_api/handlers/construction_preprocess.rs b/core-rust/mesh-api-server/src/mesh_api/handlers/construction_preprocess.rs index a9c5ef3d5e..76e978789f 100644 --- a/core-rust/mesh-api-server/src/mesh_api/handlers/construction_preprocess.rs +++ b/core-rust/mesh-api-server/src/mesh_api/handlers/construction_preprocess.rs @@ -30,6 +30,7 @@ pub(crate) async fn handle_construction_preprocess( senders.push(account); } MeshApiOperationType::Deposit => {} + MeshApiOperationType::FeePayment => {} } } diff --git a/core-rust/mesh-api-server/src/mesh_api/handlers/construction_submit.rs b/core-rust/mesh-api-server/src/mesh_api/handlers/construction_submit.rs index d751b3d9ad..94582d8a5a 100644 --- a/core-rust/mesh-api-server/src/mesh_api/handlers/construction_submit.rs +++ b/core-rust/mesh-api-server/src/mesh_api/handlers/construction_submit.rs @@ -6,19 +6,20 @@ pub(crate) async fn handle_construction_submit( ) -> Result, ResponseError> { assert_matching_network(&request.network_identifier, &state.network)?; - let (raw, intent_hash) = RawNotarizedTransaction::from_hex(&request.signed_transaction) - .ok() - .and_then(|raw| { - let tx = raw.prepare(PreparationSettings::latest_ref()); - tx.map(|tx| (raw, tx)).ok() - }) - .and_then(|(raw, tx)| Some((raw, tx.hashes().transaction_intent_hash))) - .ok_or( - ResponseError::from(ApiError::InvalidTransaction).with_details(format!( - "Invalid transaction: {}", - request.signed_transaction - )), - )?; + let raw = RawNotarizedTransaction::from_hex(&request.signed_transaction).map_err(|_| { + ResponseError::from(ApiError::InvalidTransaction).with_details(format!( + "Invalid transaction hex: {}", + &request.signed_transaction + )) + })?; + + let intent_hash = raw + .prepare(PreparationSettings::latest_ref()) + .map_err(|err| { + ResponseError::from(ApiError::InvalidTransaction) + .with_details(format!("Failed to prepare user transaction: {:?}", err)) + })? + .transaction_intent_hash(); let mempool_add_result = match state.state_manager.mempool_manager.add_and_trigger_relay( MempoolAddSource::MeshApi, diff --git a/core-rust/mesh-api-server/src/mesh_api/handlers/mempool_transaction.rs b/core-rust/mesh-api-server/src/mesh_api/handlers/mempool_transaction.rs index d75bfa32b5..949ff1caa9 100644 --- a/core-rust/mesh-api-server/src/mesh_api/handlers/mempool_transaction.rs +++ b/core-rust/mesh-api-server/src/mesh_api/handlers/mempool_transaction.rs @@ -19,27 +19,58 @@ pub(crate) async fn handle_mempool_transaction( ) .map_err(|err| err.into_response_error("intent_hash"))?; - if mempool - .get_mempool_payload_hashes_for_intent(&intent_hash) - .is_empty() - { + let payload_hashes = mempool.get_mempool_payload_hashes_for_intent(&intent_hash); + let notarized_transaction_hash = if payload_hashes.is_empty() { return Err( ResponseError::from(ApiError::TransactionNotFound).with_details(format!( - "transaction {} not found in mempool transactions", + "Transaction {} not found in mempool transactions", &request.transaction_identifier.hash )), ); - } + } else { + payload_hashes.get(0).unwrap() + }; + + let user_transaction = match mempool.get_mempool_payload(¬arized_transaction_hash) { + Some(transaction) => { + // Transaction is known to be executable, so it is safe to unwrap here + transaction.raw.into_typed().unwrap() + } + None => { + return Err( + ResponseError::from(ApiError::TransactionNotFound).with_details(format!( + "Transaction {} payload not found in mempool transactions", + &request.transaction_identifier.hash + )), + ) + } + }; + + let instructions = match user_transaction { + UserTransaction::V1(notarized_transaction) => { + notarized_transaction.signed_intent.intent.instructions.0 + } + + UserTransaction::V2(_) => { + return Err(ResponseError::from(ApiError::InvalidTransaction) + .with_details(format!("Transaction V2 not supported"))) + } + }; + + let database = state.state_manager.database.snapshot(); + let operations = to_mesh_api_operations_from_instructions_v1( + &instructions, + &mapping_context, + database.deref(), + )?; let transaction_identifier = Box::new(models::TransactionIdentifier { hash: to_api_transaction_hash_bech32m(&mapping_context, &intent_hash)?, }); - // TODO:MESH prepare transaction estimates let transaction = Box::new(models::Transaction { transaction_identifier, - // TODO:MESH Use the same approach as in `construction_parse`? - operations: vec![], + operations, related_transactions: None, metadata: None, }); diff --git a/core-rust/mesh-api-server/src/mesh_api/handlers/network_options.rs b/core-rust/mesh-api-server/src/mesh_api/handlers/network_options.rs index cc295db359..7669549520 100644 --- a/core-rust/mesh-api-server/src/mesh_api/handlers/network_options.rs +++ b/core-rust/mesh-api-server/src/mesh_api/handlers/network_options.rs @@ -8,7 +8,41 @@ pub(crate) async fn handle_network_options( let database = state.state_manager.database.snapshot(); - let mut proof_iter = database.get_proof_iter(StateVersion::pre_genesis()); + let timestamp_start_index = database + .get_proof_iter(StateVersion::pre_genesis()) + .find_map(|p| -> Option> { + // Observed that some timestamp are 0 or 1 + if p.ledger_header.proposer_timestamp_ms > 1 { + Some(to_mesh_api_block_index_from_state_version( + p.ledger_header.state_version, + )) + } else { + None + } + }) + .transpose()?; + + let ledger_header = read_current_ledger_header(database.deref()); + let previous_state_version = ledger_header.state_version.previous().map_err(|_| { + ResponseError::from(ApiError::ParentBlockNotAvailable).with_details(format!( + "Parent block not found for state version {}", + ledger_header.state_version.number() + )) + })?; + + // Attempt to scope at previous state version to check if state history is available + let historical_balance_lookup = match database.scoped_at(Some(previous_state_version)) { + Ok(_) => true, + Err(StateHistoryError::StateHistoryDisabled) => false, + Err(err) => { + return Err( + ResponseError::from(ApiError::GetStateHistoryError).with_details(format!( + "Error checking if historical balances enabled, {:?}", + err + )), + ) + } + }; // See https://docs.cdp.coinbase.com/mesh/docs/models#networkoptionsresponse for field // definitions @@ -25,18 +59,8 @@ pub(crate) async fn handle_network_options( .map(|o| o.to_string()) .collect(), errors: list_available_api_errors(), - historical_balance_lookup: false, - timestamp_start_index: proof_iter.find_map(|p| { - // Observed that some timestamp are 0 or 1 - if p.ledger_header.proposer_timestamp_ms > 1 { - Some( - to_mesh_api_block_index_from_state_version(p.ledger_header.state_version) - .unwrap(), - ) - } else { - None - } - }), + historical_balance_lookup, + timestamp_start_index, // This is for native RPC calls. Not needed for now. call_methods: vec![], balance_exemptions: vec![], diff --git a/core-rust/mesh-api-server/src/mesh_api/handlers/network_status.rs b/core-rust/mesh-api-server/src/mesh_api/handlers/network_status.rs index 0485fa73f4..a4b140ad47 100644 --- a/core-rust/mesh-api-server/src/mesh_api/handlers/network_status.rs +++ b/core-rust/mesh-api-server/src/mesh_api/handlers/network_status.rs @@ -21,7 +21,6 @@ pub(crate) async fn handle_network_status( .get_post_genesis_epoch_proof() .map(|proof| -> Result<_, MappingError> { Ok(Box::new(to_mesh_api_block_identifier_from_ledger_header( - database.deref(), &proof.ledger_header.into(), )?)) }) @@ -31,7 +30,6 @@ pub(crate) async fn handle_network_status( .get_first_proof() .map(|proof| -> Result<_, MappingError> { Ok(Box::new(to_mesh_api_block_identifier_from_ledger_header( - database.deref(), &proof.ledger_header.into(), )?)) }) @@ -44,7 +42,6 @@ pub(crate) async fn handle_network_status( .get_latest_proof() .map(|proof| -> Result<_, MappingError> { Ok(Box::new(to_mesh_api_block_identifier_from_ledger_header( - database.deref(), &proof.ledger_header.into(), )?)) }) diff --git a/core-rust/mesh-api-server/src/mesh_api/helpers.rs b/core-rust/mesh-api-server/src/mesh_api/helpers.rs index ed1023532b..c9b1a5383e 100644 --- a/core-rust/mesh-api-server/src/mesh_api/helpers.rs +++ b/core-rust/mesh-api-server/src/mesh_api/helpers.rs @@ -13,7 +13,7 @@ pub(crate) fn read_current_ledger_header( #[tracing::instrument(skip_all)] pub(crate) fn read_mandatory_main_field_substate( - database: &StateManagerDatabase, + database: &impl SubstateDatabase, node_id: &NodeId, substate_key: &SubstateKey, ) -> Result, ResponseError> { @@ -27,7 +27,7 @@ pub(crate) fn read_mandatory_main_field_substate( #[tracing::instrument(skip_all)] pub(crate) fn read_mandatory_substate( - database: &StateManagerDatabase, + database: &impl SubstateDatabase, node_id: &NodeId, partition_number: PartitionNumber, substate_key: &SubstateKey, @@ -50,7 +50,7 @@ pub(crate) fn read_mandatory_substate( } #[tracing::instrument(skip_all)] pub(crate) fn read_optional_main_field_substate( - database: &StateManagerDatabase, + database: &impl SubstateDatabase, node_id: &NodeId, substate_key: &SubstateKey, ) -> Option> { @@ -59,7 +59,7 @@ pub(crate) fn read_optional_main_field_substate( #[tracing::instrument(skip_all)] pub(crate) fn read_optional_collection_substate( - database: &StateManagerDatabase, + database: &impl SubstateDatabase, node_id: &NodeId, collection_index: CollectionIndex, substate_key: &SubstateKey, @@ -80,7 +80,7 @@ pub(crate) fn read_optional_collection_substate( #[tracing::instrument(skip_all)] pub(crate) fn read_optional_substate( - database: &StateManagerDatabase, + database: &impl SubstateDatabase, node_id: &NodeId, partition_number: PartitionNumber, substate_key: &SubstateKey, diff --git a/core-rust/mesh-api-server/src/mesh_api/metrics.rs b/core-rust/mesh-api-server/src/mesh_api/metrics.rs index d5c359e268..bb9aacbd68 100644 --- a/core-rust/mesh-api-server/src/mesh_api/metrics.rs +++ b/core-rust/mesh-api-server/src/mesh_api/metrics.rs @@ -73,7 +73,6 @@ pub struct MeshApiMetrics { } impl MeshApiMetrics { - // TODO:MESH implement it properly for MeshApi pub fn new(registry: &Registry) -> Self { MeshApiMetrics { handle_request: new_timer_vec( diff --git a/core/default.config b/core/default.config index d872a35e41..678af416c8 100644 --- a/core/default.config +++ b/core/default.config @@ -10,3 +10,6 @@ network.genesis_data=/wYAAHNOYVBwWQAHAgCfdxlxngUUXCEGCgEACQEABQkHKAAAIQoJZAAAAAr # Enable MeshAPI server api.mesh.enabled=true + +# Uncomment this to proceed with Mesh API CLI tests +#db.historical_substate_values.enable=true diff --git a/core/src/test/java/com/radixdlt/api/DeterministicMeshApiTestBase.java b/core/src/test/java/com/radixdlt/api/DeterministicMeshApiTestBase.java index d4ede19ffb..275630b80d 100644 --- a/core/src/test/java/com/radixdlt/api/DeterministicMeshApiTestBase.java +++ b/core/src/test/java/com/radixdlt/api/DeterministicMeshApiTestBase.java @@ -145,6 +145,10 @@ public Response assertErrorResponseOfType( return coreApiHelper.assertErrorResponseOfType(apiCall, responseClass); } + protected CoreApiHelper getCoreApiHelper() { + return coreApiHelper; + } + protected TransactionApi getCoreTransactionApi() { return coreApiHelper.transactionApi(); } diff --git a/core/src/test/java/com/radixdlt/api/mesh_api/MeshApiMempoolEndpointsTest.java b/core/src/test/java/com/radixdlt/api/mesh_api/MeshApiMempoolEndpointsTest.java index f2cbebe438..8a9954cf16 100644 --- a/core/src/test/java/com/radixdlt/api/mesh_api/MeshApiMempoolEndpointsTest.java +++ b/core/src/test/java/com/radixdlt/api/mesh_api/MeshApiMempoolEndpointsTest.java @@ -69,8 +69,13 @@ import com.radixdlt.api.DeterministicMeshApiTestBase; import com.radixdlt.api.core.generated.models.TransactionSubmitRequest; import com.radixdlt.api.mesh.generated.models.*; +import com.radixdlt.identifiers.Address; +import com.radixdlt.rev2.Decimal; +import com.radixdlt.rev2.Manifest; +import com.radixdlt.rev2.ScryptoConstants; import com.radixdlt.rev2.TransactionBuilder; import java.util.HashSet; +import java.util.List; import org.junit.Test; public class MeshApiMempoolEndpointsTest extends DeterministicMeshApiTestBase { @@ -81,12 +86,12 @@ public void test_mempool_endpoint() throws Exception { test.suppressUnusedWarning(); // Arrange - var expected_transaction_identifiers = new HashSet(); + var expectedTransactionIdentifiers = new HashSet(); for (int i = 0; i < 2; i++) { var transaction = TransactionBuilder.forTests().prepare(); - expected_transaction_identifiers.add( + expectedTransactionIdentifiers.add( new TransactionIdentifier() .hash(addressing.encode(transaction.transactionIntentHash()))); @@ -103,14 +108,14 @@ public void test_mempool_endpoint() throws Exception { // Act // Get mempool from the MeshAPI - var mempool_response = + var mempoolResponse = new HashSet<>( getMempoolApi() .mempool(new NetworkRequest().networkIdentifier(getNetworkIdentifier())) .getTransactionIdentifiers()); // Assert that both transactions are in the mempool list - assertThat(mempool_response.equals(expected_transaction_identifiers)); + assertThat(mempoolResponse.equals(expectedTransactionIdentifiers)); } } @@ -120,8 +125,30 @@ public void test_mempool_transaction_endpoint() throws Exception { test.suppressUnusedWarning(); // Arrange - var transaction = TransactionBuilder.forTests().prepare(); - var transaction_identifier = + var senderKeyPair = TransactionBuilder.generateKeyPair(2); + var senderAddress = Address.virtualAccountAddress(senderKeyPair.getPublicKey()); + var senderAddressStr = senderAddress.encode(networkDefinition); + + var receiverKeyPair = TransactionBuilder.generateKeyPair(3); + var receiverAddress = Address.virtualAccountAddress(receiverKeyPair.getPublicKey()); + var receiverAddressStr = receiverAddress.encode(networkDefinition); + + // Prefund sender account + getCoreApiHelper() + .submitAndWaitForSuccess(test, Manifest.depositFromFaucet(senderAddress), List.of()); + + var transaction = + TransactionBuilder.forTests() + .manifest( + Manifest.transferBetweenAccountsFeeFromSender( + senderAddress, + ScryptoConstants.XRD_RESOURCE_ADDRESS, + Decimal.ofNonNegative(1000), + receiverAddress)) + .signatories(List.of(senderKeyPair)) + .prepare(); + + var transactionIdentifier = new TransactionIdentifier().hash(addressing.encode(transaction.transactionIntentHash())); // Submit transaction to the CoreAPI @@ -136,17 +163,37 @@ public void test_mempool_transaction_endpoint() throws Exception { // Act // Get mempool transaction from the MeshAPI - var mempool_transaction_response = + var mempoolTransactionResponse = getMempoolApi() .mempoolTransaction( new MempoolTransactionRequest() .networkIdentifier(getNetworkIdentifier()) - .transactionIdentifier(transaction_identifier)) + .transactionIdentifier(transactionIdentifier)) .getTransaction(); - // Assert that transaction1 is in the mempool transaction list - assertThat(mempool_transaction_response) - .isEqualTo(new Transaction().transactionIdentifier(transaction_identifier)); + var xrdCurrency = + new Currency() + .symbol(ScryptoConstants.XRD_RESOURCE_ADDRESS.encode(networkDefinition)) + .decimals(18); + var withdrawOperation = + new Operation() + .operationIdentifier(new OperationIdentifier().index(0L)) + .type("Withdraw") + .account(new AccountIdentifier().address(senderAddressStr)) + .amount(new Amount().value("-1000000000000000000000").currency(xrdCurrency)); + var depositOperation = + new Operation() + .operationIdentifier(new OperationIdentifier().index(1L)) + .type("Deposit") + .account(new AccountIdentifier().address(receiverAddressStr)) + .amount(new Amount().value("1000000000000000000000").currency(xrdCurrency)); + var expectedTransaction = + new Transaction() + .transactionIdentifier(transactionIdentifier) + .operations(List.of(withdrawOperation, depositOperation)); + + // Assert that expected transaction is in the mempool transaction list + assertThat(mempoolTransactionResponse).isEqualTo(expectedTransaction); } } } diff --git a/testnet-node/README.md b/testnet-node/README.md index eba479aa94..88fa487ddb 100644 --- a/testnet-node/README.md +++ b/testnet-node/README.md @@ -22,9 +22,9 @@ curl \ ``` ## Running without compose -If you wanted to run without using compose you can do so like +If you wanted to run without using compose you can do so like ``` -docker run -p 127.0.0.1:3333:3333 127.0.0.1:3334:3334 127.0.0.1:3335:3335 127.0.0.1:3336:3336 -v ledger-data:/home/radixdlt/RADIXDB -v key-data:/home/radixdlt/key --env-file radix-node.env radixdlt/babylon-node:rcnet-v2-phase2-r4 +docker run -p 127.0.0.1:3333:3333 127.0.0.1:3334:3334 127.0.0.1:3335:3335 127.0.0.1:3336:3336 127.0.0.1:3337:3337 -v ledger-data:/home/radixdlt/RADIXDB -v key-data:/home/radixdlt/key --env-file radix-node.env radixdlt/babylon-node:rcnet-v2-phase2-r4 ``` ## Node volumes @@ -59,9 +59,9 @@ And then `./run.sh` in this folder. ## Debugging -It might happen that you stumble across: +It might happen that you stumble across: ``` com.sleepycat.je.DiskLimitException: (JE 18.3.12) Disk usage is not within je.maxDisk or je.freeDisk limits and write operations are prohibited: maxDiskLimit=0 freeDiskLimit=5,368,709,120 adjustedMaxDiskLimit=0 maxDiskOverage=0 freeDiskShortage=28,782,592 diskFreeSpace=5,339,926,528 availableLogSize=-28,782,592 totalLogSize=1,915,298 activeLogSize=1,915,298 reservedLogSize=0 protectedLogSize=0 protectedLogSizeMap={} ``` -This means you have (almost) reached the virtual disk memory limit of docker. You simply need to increase the limit. +This means you have (almost) reached the virtual disk memory limit of docker. You simply need to increase the limit. diff --git a/testnet-node/docker-compose.yml b/testnet-node/docker-compose.yml index 6fe529c2a7..e7842350a5 100644 --- a/testnet-node/docker-compose.yml +++ b/testnet-node/docker-compose.yml @@ -23,6 +23,7 @@ services: - "127.0.0.1:3334:3334" # System API - binds to localhost:3334 - "127.0.0.1:3335:3335" # Prometheus API - binds to localhost:3335 - "127.0.0.1:3336:3336" # Engine State API - binds to localhost:3336 + - "127.0.0.1:3337:3337" # Mesh API - binds to localhost:3337 - "127.0.0.1:9011:9011" # JMX Port for Java debugging - binds to localhost:9011 - "127.0.0.1:50505:50505" # JDWP Port for Java debugging - binds to localhost:50505 - "127.0.0.1:30000:30000" # Gossip port - binds to localhost:30000 diff --git a/testnet-node/radix-node.env b/testnet-node/radix-node.env index f94024cd5c..43d3869d6e 100644 --- a/testnet-node/radix-node.env +++ b/testnet-node/radix-node.env @@ -13,3 +13,6 @@ RADIXDLT_NETWORK_SEEDS_REMOTE=radix://node_tdx_2_1qv89yg0la2jt429vqp8sxtpg95hj63 # Enabling Engine State API's features: RADIXDLT_ENTITY_LISTING_INDICES_ENABLE=true RADIXDLT_DB_HISTORICAL_SUBSTATE_VALUES_ENABLE=true + +# Enable MeshAPI server +RADIXDLT_MESH_API_ENABLED=true