Retrieving transaction for interaction hangs

[ianthpun]

Hi there,

im having some trouble attempting to deploy on the Flow EVM.

With verbose on, this is what i see

Hardhat Ignition 🚀

Resuming existing deployment from ./ignition/deployments/chain-646

Deploying [ HelloWorldModule ]

Batch #1
  Executing HelloWorldModule#HelloWorld...

  hardhat-ignition:onchain-interaction-monitor Retrieving transaction for interaction HelloWorldModule#HelloWorld/1 from mempool (attempt 1/10) +0ms

my config looks something like this

    flow-previewnet: {
      url: "https://previewnet.evm.nodes.onflow.org",
      accounts: [PRIVATE_KEY],
      gas: 500000,
    },

The journal.jsonl shows this

{"chainId":646,"type":"DEPLOYMENT_INITIALIZE"}
{"artifactId":"HelloWorldModule#HelloWorld","constructorArgs":[],"contractName":"HelloWorld","dependencies":[],"from":"0x66146cb3f6c5078eff411b24fcbd17cc54c1657c","futureId":"HelloWorldModule#HelloWorld","futureType":"NAMED_ARTIFACT_CONTRACT_DEPLOYMENT","libraries":{},"strategy":"basic","strategyConfig":{},"type":"DEPLOYMENT_EXECUTION_STATE_INITIALIZE","value":{"_kind":"bigint","value":"0"}}
{"futureId":"HelloWorldModule#HelloWorld","networkInteraction":{"data":"0x608060405234801561001057600080fd5b506040518060400160405280600d81526020017f48656c6c6f2c20576f726c6421000000000000000000000000000000000000008152506000908051906020019061005c929190610062565b50610107565b828054600181600116156101000203166002900490600052602060002090601f016020900481019282601f106100a357805160ff19168380011785556100d1565b828001600101855582156100d1579182015b828111156100d05782518255916020019190600101906100b5565b5b5090506100de91906100e2565b5090565b61010491905b808211156101005760008160009055506001016100e8565b5090565b90565b610432806101166000396000f3fe608060405234801561001057600080fd5b50600436106100415760003560e01c806319ff1d2114610046578063d28c25d4146100c9578063ef690cc014610184575b600080fd5b61004e610207565b6040518080602001828103825283818151815260200191508051906020019080838360005b8381101561008e578082015181840152602081019050610073565b50505050905090810190601f1680156100bb5780820380516001836020036101000a031916815260200191505b509250505060405180910390f35b610182600480360360208110156100df57600080fd5b81019080803590602001906401000000008111156100fc57600080fd5b82018360208201111561010e57600080fd5b8035906020019184600183028401116401000000008311171561013057600080fd5b91908080601f016020809104026020016040519081016040528093929190818152602001838380828437600081840152601f19601f8201169050808301925050505050505091929192905050506102a9565b005b61018c6102c3565b6040518080602001828103825283818151815260200191508051906020019080838360005b838110156101cc5780820151818401526020810190506101b1565b50505050905090810190601f1680156101f95780820380516001836020036101000a031916815260200191505b509250505060405180910390f35b606060008054600181600116156101000203166002900480601f01602080910402602001604051908101604052809291908181526020018280546001816001161561010002031660029004801561029f5780601f106102745761010080835404028352916020019161029f565b820191906000526020600020905b81548152906001019060200180831161028257829003601f168201915b5050505050905090565b80600090805190602001906102bf929190610361565b5050565b60008054600181600116156101000203166002900480601f0160208091040260200160405190810160405280929190818152602001828054600181600116156101000203166002900480156103595780601f1061032e57610100808354040283529160200191610359565b820191906000526020600020905b81548152906001019060200180831161033c57829003601f168201915b505050505081565b828054600181600116156101000203166002900490600052602060002090601f016020900481019282601f106103a257805160ff19168380011785556103d0565b828001600101855582156103d0579182015b828111156103cf5782518255916020019190600101906103b4565b5b5090506103dd91906103e1565b5090565b61040391905b808211156103ff5760008160009055506001016103e7565b5090565b9056fea165627a7a7230582002dbe1e1e055e864807f7ea1c51dee2c61bc736fdd42ab10a62d88b5eb081a8d0029","id":1,"type":"ONCHAIN_INTERACTION","value":{"_kind":"bigint","value":"0"}},"type":"NETWORK_INTERACTION_REQUEST"}
{"futureId":"HelloWorldModule#HelloWorld","networkInteractionId":1,"nonce":36,"transaction":{"fees":{"gasPrice":{"_kind":"bigint","value":"0"}},"hash":"0x4712f8901e19f0fc72f93e8b7b6805efa8f483f4ba724bfb0bc13a138b1089c9"},"type":"TRANSACTION_SEND"}

txHash itself is queryable, as noted from a CURL command:

curl -X POST https://previewnet.evm.nodes.onflow.org \                           
                                                  -H "Content-Type: application/json" \
                                                  -d '{
                                                  "jsonrpc": "2.0",
                                                  "method": "eth_getTransactionByHash",
                                                  "params": ["0x4712f8901e19f0fc72f93e8b7b6805efa8f483f4ba724bfb0bc13a138b1089c9"],
                                                  "id": 1
                                              }'

I figured it must be somewhere else down the processor that its not able to save the contract address, however the lack of debug messages isn't helping me find where it is blocking

[kanej]

Hey @ianthpun, does the (attempt 1/10) increment? Or does it hang on the first attempt?

That check is that the sent transaction is in the mempool. We have a retry mechanism (with 10 retries) for infrastructure providers that load balance their mempools. So Ignition sends the transaction to the mempool and checks it has been accepted.

If the first attempt to lookup the transaction in the mempool fails that might suggest Ignition is have a problem calling the transaction lookup RPC call on the flow node.

[ianthpun]

when I first run it, it'll go through all 10 attempts and it'll return saying the transaction was dropped and that i would need to run it again

on the subsequent runs, it gets stuck at attempt 1/10, which i assume whats happening is that since its already logged with the transactionhash on the second run, it attempts to grab the transaction from the txHash but somewhere it blocks. I haven't been able to find where exactly its happening though, as that is the last debug message

[kanej]

This sounds like there is something different about the flow node.

Are you able to do a deploy against a local Hardhat node and ethereum testnet like sepolia? To eliminate your setup/Ignition module as the source.

[ianthpun]

yes correct, the deployment works on the local hardhat node, as well as Base. I suspect its someting wrong with the flow node, but its been hard to decypher just which query against the client is causing the hang

[kanej]

Can I ask you to rerun the deploy with verbose logging turned on: https://hardhat.org/hardhat-runner/docs/troubleshooting/verbose-logging

If we now which call is hanging, we can enhance the logging in Ignition to give you a better idea of what the flow node is responding with.

Are you able to do a contract deploy from a Hardhat script via either ethers or viem?

[m-Peter]

@kanej @ianthpun @nvdtf I have found the root cause of this, and it comes from this condition.
After adding some logging, I got:

hardhat-ignition:onchain-interaction-monitor Confirmations:  1 +1ms
hardhat-ignition:onchain-interaction-monitor Required Confirmations:  5 +0ms

It seems to me that this assignment:

const confirmations = block.number - receipt.blockNumber + 1;

Is always going to assign the value 1 to confirmations, as block.number will equal receipt.blockNumber. I suppose that on a live network, there will be more blocks produced, and the latest block will advance and the confirmations as well. But on a local network, or a low traffic network, this is going to hang, until there are more blocks produced. Note that Flow EVM does not have reorgs.

[kanej]

@kanej @ianthpun @nvdtf I have found the root cause of this, and it comes from this condition. After adding some logging, I got:

hardhat-ignition:onchain-interaction-monitor Confirmations:  1 +1ms
hardhat-ignition:onchain-interaction-monitor Required Confirmations:  5 +0ms

It seems to me that this assignment:

const confirmations = block.number - receipt.blockNumber + 1;

Is always going to assign the value 1 to confirmations, as block.number will equal receipt.blockNumber. I suppose that on a live network, there will be more blocks produced, and the latest block will advance and the confirmations as well. But on a local network, or a low traffic network, this is going to hang, until there are more blocks produced. Note that Flow EVM does not have reorgs.

The assumption of that code is that you are in one of two cases:

• a production chain where new blocks are being regularly minted, so eventually the number of confirmations will reach the required number of confirmations
• a development chain with automining - in which case we set the required number of confirmations to zero

What is the flow chain doing? Are blocks not minted until there is enough demand?

[m-Peter]

What is the flow chain doing? Are blocks not minted until there is enough demand?

In Flow, an EVM block is minted after an EVM transaction or a batch of EVM transactions is/are executed.
So the block minting is not regular, e.g. every 10 seconds or so, and there are no empty blocks.
Right now, Flow EVM is on PreviewNet stage, and after deploying a contract with hardhat, depending on the traffic at a given moment, the interaction will hang until there are 5 more blocks minted. But this will only happen if there are at least 5 more transactions executed, as this is what mints blocks.

Any suggestions on what can be done to work around this? The notion of confirmations on Flow EVM isn't the same as Ethereum because Flow is not using Ethereum's native consensus.

[kanej]

What is the flow chain doing? Are blocks not minted until there is enough demand?

In Flow, an EVM block is minted after an EVM transaction or a batch of EVM transactions is/are executed. So the block minting is not regular, e.g. every 10 seconds or so, and there are no empty blocks. Right now, Flow EVM is on PreviewNet stage, and after deploying a contract with hardhat, depending on the traffic at a given moment, the interaction will hang until there are 5 more blocks minted. But this will only happen if there are at least 5 more transactions executed, as this is what mints blocks.

Any suggestions on what can be done to work around this? The notion of confirmations on Flow EVM isn't the same as Ethereum because Flow is not using Ethereum's native consensus.

It sounds as if the default of waiting for 5 confirmations is inappropriate for flow. You could set the number of confirmations to 1, to avoid waiting after the deployment transaction is minted into a block. The required confirmations config setting is covered here:

https://hardhat.org/ignition/docs/config#requiredconfirmations

[m-Peter]

@kanej You the best 🙌

All I had to do is:

ignition: {
  requiredConfirmations: 1,
},

And it worked perfectly:

➜  hardhat-testing npx hardhat ignition deploy ./ignition/modules/Apollo.js  --network flowevm --verbose --show-stack-traces

Hardhat Ignition 🚀

Deploying [ Apollo ]

Batch #1
  Executing Apollo#Rocket...

  hardhat-ignition:onchain-interaction-monitor Retrieving transaction for interaction Apollo#Rocket/1 from mempool (attempt 1/10) +0ms
  hardhat-ignition:onchain-interaction-monitor Transaction lookup for Apollo#Rocket/1 not found in mempool, waiting 1000 sBatch #1
  Executed Apollo#Rocket

Batch #2
  Executing Apollo#Rocket.launch...

  hardhat-ignition:onchain-interaction-monitor Retrieving transaction for interaction Apollo#Rocket.launch/1 from mempool (attempt 1/10) +229ms
  hardhat-ignition:onchain-interaction-monitor Transaction lookup for Apollo#Rocket.launch/1 not found in mempool, waitingBatch #2
  Executed Apollo#Rocket.launch

[ Apollo ] successfully deployed 🚀

Deployed Addresses

Apollo#Rocket - 0x50aC67b41B4610657F895f2f80a3Fc2816ca172C
  hardhat:core:cli Killing Hardhat after successfully running task deploy +0ms

[kanej]

I am going to close this issue with the advice that flow requires setting the required confirmations to 1 to avoid hanging until 5 further blocks have been mined:

ignition: {
  requiredConfirmations: 1,
},