Skip to content

Commit

Permalink
update to use 2d array vs jsonb
Browse files Browse the repository at this point in the history
Signed-off-by: Jesse Nelson <[email protected]>
  • Loading branch information
jnels124 committed Feb 28, 2025
1 parent 6f06a3b commit 0adfa57
Showing 1 changed file with 29 additions and 33 deletions.
62 changes: 29 additions & 33 deletions docs/design/batch-transactions.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,22 +22,12 @@ This document details how the mirror node will be updated to support it.

```postgresql
alter table if exists transaction
add column if not exists batch_key bytea default null,
add column if not exists inner_transactions jsonb default null;
add column if not exists batch_key bytea default null,
add column if not exists inner_transactions bigint[][] default null;
```

### Importer

Create

```java
public record InnerTransaction(
long payerAccountId,
long validStartNs
) {
}
```

Update

```java
Expand All @@ -50,23 +40,20 @@ Update

```java
public class Transaction {
private byte[] batchKey;
private byte[] batchKey;
private List<long[]> innerTransactions;

@JsonSerialize(using = ObjectToStringSerializer.class)
@JdbcTypeCode(SqlTypes.JSON)
private List<InnerTransaction> innerTransactions;

public void addInnerTransaction(Transaction transaction) {
if (innerTransactions == null) {
innerTransactions = new ArrayList<>();
}

innerTransactions.add(transaction.toInnerTransaction());
public void addInnerTransaction(Transaction transaction) {
if (innerTransactions == null) {
innerTransactions = new ArrayList<>();
}

private InnerTransaction toInnerTransaction() {
return new InnerTransaction(payerAccountId, validStartNs);
}
innerTransactions.add(transaction.toInnerTransaction());
}

private long[] toInnerTransaction() {
return new long []{payerAccountId, validStartNs};
}

}
```
Expand Down Expand Up @@ -94,14 +81,14 @@ public class SqlEntityListener implements EntityListener, RecordStreamFileListen
if (transaction.getBatchKey() != null && transaction.getNonce() == 0) {
Transaction batchParent = context.get(Transaction.class, transaction.getParentConsensus());

if (batchParent == null) {
throw new ImporterException("Batch parent not found for transaction: " + transaction.getConsensusTimestamp());
}
if (batchParent == null) {
throw new ImporterException("Batch parent not found for transaction: " + transaction.getConsensusTimestamp());
}

batchParent.addInnerTransaction(transaction);
}
// ...continue with current logic
batchParent.addInnerTransaction(transaction);
}
// ...continue with current logic
}
}
```

Expand Down Expand Up @@ -130,7 +117,7 @@ public class SqlEntityListener implements EntityListener, RecordStreamFileListen
- Update `Transactions.getTransactionsByIdOrHash` to query all `payer_account_id` and `valid_start_ns` combinations
contained in `transaction.inner_transactions` for transaction(s) matching the `transactionId`

```psudo
```pseudo
const getTransactionsByIdOrHash = async (req, res) => {
If request by transactionId
Get Transactions by transactionId using current logic
Expand Down Expand Up @@ -181,6 +168,15 @@ INNER_TRANSACTION_FAILED
- Retrieving transactions by transaction id should have no performance impact for non batch transactions and minimal
performance impact for batch transactions

## Rejected Ideas

- Store inner transactions (with all transaction columns) in a separate table with additional column for batch payer.
This would allow colocating inner transactions with their batch parent, but would require additional
storage for duplicated data. Only downside to not taking this approach is additional query overhead when
retrieving batch transactions by transaction id.
- Store inner transactions as a jsonb column on transaction with array of payer account and valid start. This approach
was ejected due to potential bloat from using jsonb data type.

## Open Questions

- Can block streams remove duplication of inner transaction bytes?

0 comments on commit 0adfa57

Please sign in to comment.