catch_and_close Stream overrun

[cppfuns]

Run the fill-pg command for the first time as follows

./fill-pg --fill-connect-to 127.0.0.1:8080 --fpg-create

Check PostgreSQL at this time, the corresponding table has been created

But after checking the terminal returns an error
The data in the table is not written

info  2020-12-24T06:45:17.195 fill-pg   main.cpp:98                   main                 ] fill-pg version v0.3.0-alpha-26-gf415e46
info  2020-12-24T06:45:17.195 fill-pg   main.cpp:99                   main                 ] fill-pg using configuration file /root/.local/share/eosio/fill-pg/config/config.ini
info  2020-12-24T06:45:17.195 fill-pg   main.cpp:100                  main                 ] fill-pg data directory is /root/.local/share/eosio/fill-pg/data
info  2020-12-24T06:45:17.195 fill-pg   fill_pg_plugin.cpp:98         fpg_session          ] connect to postgresql
info  2020-12-24T06:45:17.927 fill-pg   state_history_connecti:63     connect              ] connect to 127.0.0.1:8080
error 2020-12-24T06:45:17.928 fill-pg   state_history_connecti:168    catch_and_close      ] Stream overrun
info  2020-12-24T06:45:17.928 fill-pg   state_history_connecti:193    close                ] closing state-history socket

node version:v2.0.7
os: ubuntu 18.04

Related parameters in config.ini

state-history-dir = "state-history"
trace-history = true
chain-state-history = true
plugin = eosio::state_history_plugin
state-history-endpoint = 0.0.0.0:8080

[cppfuns]

@tbfleming @larryk85

[jgiszczak]

@cppfuns Please try again with the current develop branch of history-tools. We have been resolving compatibility issues there.

[garden2weini]

@cppfuns Please try again with the current develop branch of history-tools. We have been resolving compatibility issues there.

I clone develop branch of history-tools, and buld a docker image using ubuntu-18.04.dockerfile. But the same error still occur.

root@iZj6c0lvzs7170x7omf10oZ:~/history-tools/build# ./fill-pg --fill-connect-to 127.0.0.1:8080
info  2021-03-12T09:23:05.178 fill-pg   main.cpp:98                   main                 ] fill-pg version Unknown
info  2021-03-12T09:23:05.178 fill-pg   main.cpp:99                   main                 ] fill-pg using configuration file /root/.local/share/eosio/fill-pg/config/config.ini
info  2021-03-12T09:23:05.179 fill-pg   main.cpp:100                  main                 ] fill-pg data directory is /root/.local/share/eosio/fill-pg/data
info  2021-03-12T09:23:05.180 fill-pg   fill_pg_plugin.cpp:98         fpg_session          ] connect to postgresql
info  2021-03-12T09:23:05.203 fill-pg   state_history_connecti:63     connect              ] connect to 127.0.0.1:8080
error 2021-03-12T09:23:05.214 fill-pg   state_history_connecti:168    catch_and_close      ] Stream overrun
info  2021-03-12T09:23:05.215 fill-pg   state_history_connecti:193    close                ] closing state-history socket

[elmato]

Having the same problem with:

• ship from eosio_2.1.0-rc3-ubuntu-18.04_amd64.deb
• history-tools at c763c19

[garden2weini]

I pull eos and hitstory-tools from dockerhub, then can run history tools.
docker pull eosio/eosio:release_2.1.x
docker pull eosio/history-tools:c763c194d5275de6d10175a9fc02342ea57560d8

[elmato]

@elmato I'd like to test my tiny fix in a various environments, so try one more time pls with patch from #104

Having the same problem with:

• ship from eosio_2.1.0-rc3-ubuntu-18.04_amd64.deb
• history-tools at c763c19

Thanks @633kh4ck, I tried it yesterday but with no luck, im still getting the same "Stream overrun" error as soon as fill-pg starts

[elmato]

@elmato I'd like to test my tiny fix in a various environments, so try one more time pls with patch from #104

Having the same problem with:

• ship from eosio_2.1.0-rc3-ubuntu-18.04_amd64.deb
• history-tools at c763c19

Thanks @633kh4ck, I tried it yesterday but with no luck, im still getting the same "Stream overrun" error as soon as fill-pg starts

Just for clarification: you need to build fill-pg (a-ka history-tools) from my fork (at 633kh4ck@7c251e4) because patch (#104) hasn't been merged yet

Building fill-pg from your branch and running it against ship from eosio_2.1.0-rc3-ubuntu-18.04_amd64.deb I'm getting a different error.

Bad variant index

➜  build git:(fix/safe-deltas-unpack) nodeos --version
v2.1.0-rc3
➜  build git:(fix/safe-deltas-unpack) ./fill-pg --fpg-drop --fpg-create --fill-connect-to=127.0.0.1:5432
info  2021-03-15T19:24:22.935 fill-pg   main.cpp:98                   main                 ] fill-pg version v0.3.0-alpha-27-g7c251e4
info  2021-03-15T19:24:22.935 fill-pg   main.cpp:99                   main                 ] fill-pg using configuration file /home/me/.local/share/eosio/fill-pg/config/config.ini
info  2021-03-15T19:24:22.935 fill-pg   main.cpp:100                  main                 ] fill-pg data directory is /home/me/.local/share/eosio/fill-pg/data
info  2021-03-15T19:24:22.935 fill-pg   fill_pg_plugin.cpp:98         fpg_session          ] connect to postgresql
info  2021-03-15T19:24:22.947 fill-pg   fill_pg_plugin.cpp:105        start                ] drop schema "chain"
info  2021-03-15T19:24:22.947 fill-pg   state_history_connecti:63     connect              ] connect to 127.0.0.1:5432
info  2021-03-15T19:24:22.950 fill-pg   fill_pg_plugin.cpp:230        create_tables        ] create schema "chain"
error 2021-03-15T19:24:23.073 fill-pg   state_history_connecti:168    catch_and_close      ] Bad variant index
info  2021-03-15T19:24:23.073 fill-pg   state_history_connecti:193    close                ] closing state-history socket

[garden2weini]

@garden2weini You just pulled the docker images (the commands you specified) - it has nothing to do with their actual usage. Have you tested them? Can you provide us with real details and logs pls?

I pull eos and hitstory-tools from dockerhub, then can run history tools.
docker pull eosio/eosio:release_2.1.x
docker pull eosio/history-tools:c763c194d5275de6d10175a9fc02342ea57560d8

you can ref https://github.com/garden2weini/eos-state-history-client/blob/master/History-Tools.md

EOS logs

info  2021-03-16T03:32:59.857 nodeos    state_history_plugin.c:237    send_update          ] pushing result {"head":{"block_num":165524202},"last_irreversible":{"block_num":165523869},"this_block":{"block_num":165208036}} to send queue
info  2021-03-16T03:32:59.963 nodeos    state_history_plugin.c:237    send_update          ] pushing result {"head":{"block_num":165524202},"last_irreversible":{"block_num":165523869},"this_block":{"block_num":165208037}} to send queue
info  2021-03-16T03:33:00.039 nodeos    state_history_plugin.c:237    send_update          ] pushing result {"head":{"block_num":165524202},"last_irreversible":{"block_num":165523869},"this_block":{"block_num":165208038}} to send queue
info  2021-03-16T03:33:00.183 nodeos    state_history_plugin.c:237    send_update          ] pushing result {"head":{"block_num":165524202},"last_irreversible":{"block_num":165523869},"this_block":{"block_num":165208039}} to send queue
info  2021-03-16T03:33:00.396 nodeos    state_history_plugin.c:237    send_update          ] pushing result {"head":{"block_num":165524202},"last_irreversible":{"block_num":165523869},"this_block":{"block_num":165208040}} to send queue
info  2021-03-16T03:33:00.524 nodeos    state_history_plugin.c:237    send_update          ] pushing result {"head":{"block_num":165524202},"last_irreversible":{"block_num":165523869},"this_block":{"block_num":165208041}} to send queue
info  2021-03-16T03:33:00.742 nodeos    state_history_plugin.c:237    send_update          ] pushing result {"head":{"block_num":165524202},"last_irreversible":{"block_num":165523869},"this_block":{"block_num":165208042}} to send queue
info  2021-03-16T03:33:01.155 nodeos    state_history_plugin.c:237    send_update          ] pushing result {"head":{"block_num":165524204},"last_irreversible":{"block_num":165523869},"this_block":{"block_num":165208043}} to send queue
info  2021-03-16T03:33:03.192 nodeos    state_history_plugin.c:237    send_update          ] pushing result {"head":{"block_num":165524207},"last_irreversible":{"block_num":165523881},"this_block":{"block_num":165208044}} to send queue
info  2021-03-16T03:33:24.314 nodeos    state_history_plugin.c:237    send_update          ] pushing result {"head":{"block_num":165524253},"last_irreversible":{"block_num":165523917},"this_block":{"block_num":165208045}} to send queue
info  2021-03-16T03:33:26.955 net-18    net_plugin.cpp:3425           connection_monitor   ] p2p client connections: 0/3, peer connections: 12/8

History Tools logs

info  2021-03-16T02:35:19.328 fill-pg   main.cpp:98                   main                 ] fill-pg version Unknown
info  2021-03-16T02:35:19.329 fill-pg   main.cpp:99                   main                 ] fill-pg using configuration file /root/.local/share/eosio/fill-pg/config/config.ini
info  2021-03-16T02:35:19.330 fill-pg   main.cpp:100                  main                 ] fill-pg data directory is /root/.local/share/eosio/fill-pg/data
info  2021-03-16T02:35:19.332 fill-pg   fill_pg_plugin.cpp:98         fpg_session          ] connect to postgresql
info  2021-03-16T02:35:19.379 fill-pg   state_history_connecti:63     connect              ] connect to 127.0.0.1:8080
info  2021-03-16T02:36:00.478 fill-pg   fill_pg_plugin.cpp:512        received             ] switch forks at block 165206199
info  2021-03-16T02:36:00.478 fill-pg   fill_pg_plugin.cpp:521        received             ] block 165206199
info  2021-03-16T02:43:19.222 fill-pg   fill_pg_plugin.cpp:646        close_streams        ] block 165206200 - 165206399
info  2021-03-16T02:47:42.615 fill-pg   fill_pg_plugin.cpp:646        close_streams        ] block 165206400 - 165206599
info  2021-03-16T02:48:52.795 fill-pg   fill_pg_plugin.cpp:646        close_streams        ] block 165206600 - 165206799
info  2021-03-16T02:52:15.600 fill-pg   fill_pg_plugin.cpp:646        close_streams        ] block 165206800 - 165206999
info  2021-03-16T02:53:00.890 fill-pg   fill_pg_plugin.cpp:646        close_streams        ] block 165207000 - 165207199
info  2021-03-16T02:54:06.232 fill-pg   fill_pg_plugin.cpp:646        close_streams        ] block 165207200 - 165207399
info  2021-03-16T03:10:22.263 fill-pg   fill_pg_plugin.cpp:646        close_streams        ] block 165207400 - 165207599
info  2021-03-16T03:23:59.790 fill-pg   fill_pg_plugin.cpp:646        close_streams        ] block 165207600 - 165207799
info  2021-03-16T03:29:18.873 fill-pg   fill_pg_plugin.cpp:646        close_streams        ] block 165207800 - 165207999

[pocrd]

How to fix this issue in latest version?