From e80763fe1349be659594f3a21ac76b9608ee4fe9 Mon Sep 17 00:00:00 2001 From: moisses89 <7888669+moisses89@users.noreply.github.com> Date: Thu, 28 Sep 2023 12:08:40 +0200 Subject: [PATCH 1/3] Add blockchain indexing section --- safe-core-api/safe-transaction-service.md | 34 ++++++++++++++++++++++- 1 file changed, 33 insertions(+), 1 deletion(-) diff --git a/safe-core-api/safe-transaction-service.md b/safe-core-api/safe-transaction-service.md index bcf71552..53ca8218 100644 --- a/safe-core-api/safe-transaction-service.md +++ b/safe-core-api/safe-transaction-service.md @@ -5,7 +5,7 @@ Safe transaction service keeps track of transactions sent via Safe contracts. It **Key Features:** -- [**Blockchain Indexing**](): Executed transactions, configuration changes, ERC20/721 transfers, and onchain confirmations are automatically indexed from the blockchain. +- [**Blockchain Indexing**](#blockchain-indexing): Executed transactions, configuration changes, ERC20/721 transfers, and onchain confirmations are automatically indexed from the blockchain. - [**Offchain transaction signatures**](#offchain-transaction-signatures): Transactions can be sent to the service, enabling offchain signature collection. This feature helps inform owners about pending transactions that are awaiting confirmation to be executed. - [**Offchain messages**](#offchain-messages): The service can collect offchain signatures to confirm messages following [EIP-1271](https://eips.ethereum.org/EIPS/eip-1271). - [**Transactions decode**](#transactions-decode): The service keeps getting source and ABIs from contracts that interact with Safe to decode these interactions. @@ -23,6 +23,38 @@ Safe transaction service is a [Django](https://www.djangoproject.com/) app writt
+## Blockchain Indexing +Safe transaction service can index automatically executed transactions, configuration changes, ERC20/721 transfers, and onchain confirmations. +The indexer is running on `worker-indexer` by different periodic [tasks](https://github.com/safe-global/safe-transaction-service/blob/master/safe_transaction_service/history/tasks.py). + +ERC20 and ERC721 are indexed using [eth_getLogs](https://ethereum.org/en/developers/docs/apis/json-rpc/#eth_getlogs) filtered by the Transfer topic `keccak('Transfer(address,address,uint256)')`. + +Safe creation, executed transactions, configuration changes and onchain confirmations are indexed at different way depending if the chain is L1 or L2. + +For L1 chains the indexer call tracing methods, for oldest blocks [trace_filter](https://openethereum.github.io/JSONRPC-trace-module#trace_filter) is used filtered by singleton address of safe contracts and for latest blocks [trace_block](https://openethereum.github.io/JSONRPC-trace-module#trace_block). The number of considered latest blocks is defined in `ETH_INTERNAL_TXS_NUMBER_TRACE_BLOCKS`. The environment variables used by indexing are defined [here](https://github.com/safe-global/safe-transaction-service/blob/master/config/settings/base.py#L433). + +For L2 chains the indexing is by events with [eth_getLogs](https://ethereum.org/en/developers/docs/apis/json-rpc/#eth_getlogs) method with the corresponding topics. + +From safe creation the transaction service is storing each contract change on `SafeStatus` model as nonce, owners... and the last current status of a Safe in `SafeLastStatus`. + +The following endpoints let us to know the current indexing status on safe transaction service: +- `/v1/about/indexing/` + +Response example: +```json +{ + "currentBlockNumber": 9773327, // Last block on blockchain + "erc20BlockNumber": 9773326, // Last block indexed for erc20/721 events + "erc20Synced": true, + "masterCopiesBlockNumber": 9773327, // Last block indexed for executed transactions, ether transfers, configuration changes... + "masterCopiesSynced": true, + "synced": true +} +``` + +**Reorgs handling** +When indexed every block is marked as not confirmed unless it has some depth (configured via ETH_REORG_BLOCKS environment variable). Not confirmed blocks are checked periodically to check if the blockchain blockHash for that number changed before it reaches the desired number of confirmations, if that's the case, all blocks from that block and the transactions related are deleted and indexing is restarted to the last confirmed block. + ## Offchain Transaction Signatures The transaction service can collect offchain transaction signatures, allowing the owners to share their signatures to reach the required threshold before executing a transaction and spending less gas than onchain approvals. From 4e19e50fb39e0cb444ec675d6c2429e990266cf3 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Mois=C3=A9s=20Fern=C3=A1ndez?= <7888669+moisses89@users.noreply.github.com> Date: Thu, 28 Sep 2023 14:03:29 +0200 Subject: [PATCH 2/3] Apply suggestions from code review MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Uxío --- safe-core-api/safe-transaction-service.md | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/safe-core-api/safe-transaction-service.md b/safe-core-api/safe-transaction-service.md index 53ca8218..286820f2 100644 --- a/safe-core-api/safe-transaction-service.md +++ b/safe-core-api/safe-transaction-service.md @@ -31,11 +31,11 @@ ERC20 and ERC721 are indexed using [eth_getLogs](https://ethereum.org/en/develop Safe creation, executed transactions, configuration changes and onchain confirmations are indexed at different way depending if the chain is L1 or L2. -For L1 chains the indexer call tracing methods, for oldest blocks [trace_filter](https://openethereum.github.io/JSONRPC-trace-module#trace_filter) is used filtered by singleton address of safe contracts and for latest blocks [trace_block](https://openethereum.github.io/JSONRPC-trace-module#trace_block). The number of considered latest blocks is defined in `ETH_INTERNAL_TXS_NUMBER_TRACE_BLOCKS`. The environment variables used by indexing are defined [here](https://github.com/safe-global/safe-transaction-service/blob/master/config/settings/base.py#L433). +For L1 chains the indexer calls tracing methods, for oldest blocks [trace_filter](https://openethereum.github.io/JSONRPC-trace-module#trace_filter) is used filtering by singleton address of Safe contracts and for latest blocks [trace_block](https://openethereum.github.io/JSONRPC-trace-module#trace_block) is used, as `trace_filter` takes longer to return updated information. `trace_block` will be used if the block depth is lower than `ETH_INTERNAL_TXS_NUMBER_TRACE_BLOCKS`. The environment variables used by indexing are defined [here](https://github.com/safe-global/safe-transaction-service/blob/master/config/settings/base.py#L433). For L2 chains the indexing is by events with [eth_getLogs](https://ethereum.org/en/developers/docs/apis/json-rpc/#eth_getlogs) method with the corresponding topics. -From safe creation the transaction service is storing each contract change on `SafeStatus` model as nonce, owners... and the last current status of a Safe in `SafeLastStatus`. +From Safe creation the transaction service is storing each contract change on `SafeStatus` model as `nonce`, `owners`... and the last current status of a Safe in `SafeLastStatus`, for database easy access and optimization. The following endpoints let us to know the current indexing status on safe transaction service: - `/v1/about/indexing/` @@ -52,11 +52,13 @@ Response example: } ``` -**Reorgs handling** -When indexed every block is marked as not confirmed unless it has some depth (configured via ETH_REORG_BLOCKS environment variable). Not confirmed blocks are checked periodically to check if the blockchain blockHash for that number changed before it reaches the desired number of confirmations, if that's the case, all blocks from that block and the transactions related are deleted and indexing is restarted to the last confirmed block. +### Reorgs handling +Every block is marked as `not confirmed` during indexing unless it has some depth (configured via `ETH_REORG_BLOCKS` environment variable). Not confirmed blocks are checked periodically to check if the blockchain `blockHash` for that number changed before it reaches the desired number of confirmations, if that's the case, all blocks from that block and the transactions related are deleted and indexing is restarted to the last confirmed block. + +**Note:** No offchain signatures, transactions or messages are lost on this process. Only onchain data is removed. ## Offchain Transaction Signatures -The transaction service can collect offchain transaction signatures, allowing the owners to share their signatures to reach the required threshold before executing a transaction and spending less gas than onchain approvals. +Safe Transaction Service can collect offchain transaction signatures, allowing the owners to share their signatures to reach the required threshold before executing a transaction and spending less gas than onchain approvals. The following endpoints let us propose a transaction and collect every confirmation (offchain signatures): From 7bfccf78301d56f3738be775124258ec404a70d6 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Mois=C3=A9s=20Fern=C3=A1ndez?= <7888669+moisses89@users.noreply.github.com> Date: Mon, 2 Oct 2023 10:36:49 +0200 Subject: [PATCH 3/3] Apply suggestions from code review Co-authored-by: Tanay Pant <7481165+tanay1337@users.noreply.github.com> --- safe-core-api/safe-transaction-service.md | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/safe-core-api/safe-transaction-service.md b/safe-core-api/safe-transaction-service.md index 286820f2..0a52857a 100644 --- a/safe-core-api/safe-transaction-service.md +++ b/safe-core-api/safe-transaction-service.md @@ -5,7 +5,7 @@ Safe transaction service keeps track of transactions sent via Safe contracts. It **Key Features:** -- [**Blockchain Indexing**](#blockchain-indexing): Executed transactions, configuration changes, ERC20/721 transfers, and onchain confirmations are automatically indexed from the blockchain. +- [**Blockchain Indexing**](#blockchain-indexing): Executed transactions, configuration changes, ERC-20/721 transfers, and onchain confirmations are automatically indexed from the blockchain. - [**Offchain transaction signatures**](#offchain-transaction-signatures): Transactions can be sent to the service, enabling offchain signature collection. This feature helps inform owners about pending transactions that are awaiting confirmation to be executed. - [**Offchain messages**](#offchain-messages): The service can collect offchain signatures to confirm messages following [EIP-1271](https://eips.ethereum.org/EIPS/eip-1271). - [**Transactions decode**](#transactions-decode): The service keeps getting source and ABIs from contracts that interact with Safe to decode these interactions. @@ -24,20 +24,20 @@ Safe transaction service is a [Django](https://www.djangoproject.com/) app writt
## Blockchain Indexing -Safe transaction service can index automatically executed transactions, configuration changes, ERC20/721 transfers, and onchain confirmations. +Safe transaction service can index automatically executed transactions, configuration changes, ERC-20/721 transfers, and onchain confirmations. The indexer is running on `worker-indexer` by different periodic [tasks](https://github.com/safe-global/safe-transaction-service/blob/master/safe_transaction_service/history/tasks.py). -ERC20 and ERC721 are indexed using [eth_getLogs](https://ethereum.org/en/developers/docs/apis/json-rpc/#eth_getlogs) filtered by the Transfer topic `keccak('Transfer(address,address,uint256)')`. +ERC-20 and ERC-721 are indexed using [eth_getLogs](https://ethereum.org/en/developers/docs/apis/json-rpc/#eth_getlogs) filtered by the Transfer topic `keccak('Transfer(address,address,uint256)')`. -Safe creation, executed transactions, configuration changes and onchain confirmations are indexed at different way depending if the chain is L1 or L2. +Safe creation, executed transactions, configuration changes, and onchain confirmations are indexed differently depending on whether the chain is L1 or L2. -For L1 chains the indexer calls tracing methods, for oldest blocks [trace_filter](https://openethereum.github.io/JSONRPC-trace-module#trace_filter) is used filtering by singleton address of Safe contracts and for latest blocks [trace_block](https://openethereum.github.io/JSONRPC-trace-module#trace_block) is used, as `trace_filter` takes longer to return updated information. `trace_block` will be used if the block depth is lower than `ETH_INTERNAL_TXS_NUMBER_TRACE_BLOCKS`. The environment variables used by indexing are defined [here](https://github.com/safe-global/safe-transaction-service/blob/master/config/settings/base.py#L433). +For L1 chains, the indexer calls tracing methods. For the oldest blocks, [trace_filter](https://openethereum.github.io/JSONRPC-trace-module#trace_filter) is used filtering by singleton address of Safe contracts, and for the latest blocks [trace_block](https://openethereum.github.io/JSONRPC-trace-module#trace_block) is used, as `trace_filter` takes longer to return updated information. `trace_block` will be used if the block depth is lower than `ETH_INTERNAL_TXS_NUMBER_TRACE_BLOCKS`. The environment variables indexing uses are defined [here](https://github.com/safe-global/safe-transaction-service/blob/master/config/settings/base.py#L433). -For L2 chains the indexing is by events with [eth_getLogs](https://ethereum.org/en/developers/docs/apis/json-rpc/#eth_getlogs) method with the corresponding topics. +For L2 chains, the indexing is by events with the [eth_getLogs](https://ethereum.org/en/developers/docs/apis/json-rpc/#eth_getlogs) method with the corresponding topics. -From Safe creation the transaction service is storing each contract change on `SafeStatus` model as `nonce`, `owners`... and the last current status of a Safe in `SafeLastStatus`, for database easy access and optimization. +From Safe creation, the transaction service stores each contract change on the `SafeStatus` model as `nonce`, `owners`, etc. The latest and current status of a Safe is stored as `SafeLastStatus` for easy database access and optimization. -The following endpoints let us to know the current indexing status on safe transaction service: +The following endpoints let us know the current indexing status of the Safe transaction service: - `/v1/about/indexing/` Response example: @@ -46,16 +46,16 @@ Response example: "currentBlockNumber": 9773327, // Last block on blockchain "erc20BlockNumber": 9773326, // Last block indexed for erc20/721 events "erc20Synced": true, - "masterCopiesBlockNumber": 9773327, // Last block indexed for executed transactions, ether transfers, configuration changes... + "masterCopiesBlockNumber": 9773327, // Last block indexed for executed transactions, ether transfers, configuration changes, etc. "masterCopiesSynced": true, "synced": true } ``` ### Reorgs handling -Every block is marked as `not confirmed` during indexing unless it has some depth (configured via `ETH_REORG_BLOCKS` environment variable). Not confirmed blocks are checked periodically to check if the blockchain `blockHash` for that number changed before it reaches the desired number of confirmations, if that's the case, all blocks from that block and the transactions related are deleted and indexing is restarted to the last confirmed block. +Every block is marked as `not confirmed` during indexing unless it has some depth (configured via the `ETH_REORG_BLOCKS` environment variable). Unconfirmed blocks are checked periodically to see if the blockchain `blockHash` for that number changed before it reaches the desired number of confirmations. If that's the case, all blocks from that block and related transactions are deleted, and indexing is restarted to the last confirmed block. -**Note:** No offchain signatures, transactions or messages are lost on this process. Only onchain data is removed. +**Note:** No offchain signatures, transactions, or messages are lost in this process. Only onchain data is removed. ## Offchain Transaction Signatures Safe Transaction Service can collect offchain transaction signatures, allowing the owners to share their signatures to reach the required threshold before executing a transaction and spending less gas than onchain approvals.