Tapd stuck and not communicating with tapcli

[snow884]

I tried minting a number of new assets . At one point tapd appeared to get stuck and stopped responding to requests from tapcli .

tapd logs:

2023-07-05 18:32:12.511 [DBG] GRDN: Inspecting tx f0a2926add9e17787a3f3b4ecd48837d0bf56d54dcc1858bdf623fe5d4c28f67 for Taproot Asset address outputs
2023-07-05 18:32:12.512 [DBG] GRDN: Inspecting tx 263fe0af38ae17fec2ce2304c7727b3443fb9503750ba877568788019703efbb for Taproot Asset address outputs
2023-07-05 18:32:12.512 [DBG] GRDN: Inspecting tx 804baf6eb869774321ea9aaff40d5429306e29f6c7b18e3ec4ea3786849a48e1 for Taproot Asset address outputs
2023-07-05 18:32:12.512 [INF] GRDN: Starting main custodian event loop
2023-07-05 18:32:12.515 [INF] GRDN: BatchCaretaker(021507534593d69fe6efea1aa225879bbd6efe0f3f786ee8f192204b1e18f34630): transition states: BatchStateCommitted -> BatchStateBroadcast
2023-07-05 18:32:12.515 [INF] GRDN: BatchCaretaker(021507534593d69fe6efea1aa225879bbd6efe0f3f786ee8f192204b1e18f34630): extracted finalized GenesisTx
2023-07-05 18:32:12.522 [DBG] GRDN: Inspecting tx 68f1bc3cb7a19974462a1cba83de8bc878a3f3c97b1a853e7f491d9f1a4908d8 for Taproot Asset address outputs
2023-07-05 18:32:12.531 [INF] GRDN: BatchCaretaker(021507534593d69fe6efea1aa225879bbd6efe0f3f786ee8f192204b1e18f34630): transition states: BatchStateBroadcast -> BatchStateBroadcast
2023-07-05 18:32:12.969 [INF] PROF: Attempting to receive proof via sid=72daea4bc3abf0f6d6a667615c5dfc6289ad437eedea7c23cf5d704862353c21ba7e8b97999c7368e2aaa7f3e695d135fb804449f15e5ae4afe5237b5269cf62
2023-07-05 18:37:27.294 [DBG] GRDN: Inspecting tx 68f1bc3cb7a19974462a1cba83de8bc878a3f3c97b1a853e7f491d9f1a4908d8 for Taproot Asset address outputs
2023-07-05 18:37:27.306 [DBG] GRDN: Got chain confirmation: 68f1bc3cb7a19974462a1cba83de8bc878a3f3c97b1a853e7f491d9f1a4908d8
2023-07-05 18:37:27.306 [INF] GRDN: MintingBatch(021507534593d69fe6efea1aa225879bbd6efe0f3f786ee8f192204b1e18f34630): confirmed at block(hash=00000000000000162606d660ad5ab09562c5018dfb779f930164c236a1c9358c, height=2440527)
2023-07-05 18:37:27.306 [INF] GRDN: BatchCaretaker(021507534593d69fe6efea1aa225879bbd6efe0f3f786ee8f192204b1e18f34630), advancing from state=BatchStateConfirmed to state=BatchStateFinalized
2023-07-05 18:37:27.309 [DBG] GRDN: Registering asset with universe, key=(universe.Identifier) f6cd8c2ae720261eff3dca8d04cee5f30bf1d48d234032aea750a6323cab5581

tapcli logs:

(base) [ec2-user@ip-172-31-91-154 ~]$ tapcli assets list
[tapcli] unable to list assets: rpc error: code = Unavailable desc = connection error: desc = "transport: authentication handshake failed: context deadline exceeded"
(base) [ec2-user@ip-172-31-91-154 ~]$ tapcli assets list
[tapcli] unable to list assets: rpc error: code = Unavailable desc = connection error: desc = "transport: authentication handshake failed: read tcp 127.0.0.1:48154->127.0.0.1:10029: i/o timeout"
(base) [ec2-user@ip-172-31-91-154 ~]$ tapcli assets list
[tapcli] unable to list assets: rpc error: code = Unavailable desc = connection error: desc = "transport: authentication handshake failed: read tcp 127.0.0.1:51400->127.0.0.1:10029: i/o timeout"

[ffranr]

It looks as if tapd might be too busy (registering a minting event with a universe?) to accept further requests. I would be interested in seeing more tapd logs once they are produced.

[Roasbeef]

Are you able to get a profile of right when it starts to hang?

[guggero]

Likely fixed by #431. Please re-open if able to reproduce.