Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add sequencer data to stream server #2517

Merged
merged 35 commits into from
Sep 19, 2023
Merged

Add sequencer data to stream server #2517

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
4a89b44
add data stream server to sequencer
ToniRamirezM Sep 6, 2023
6a73402
update docker-compose
ToniRamirezM Sep 6, 2023
ece27bc
add sequencer data to stream server
ToniRamirezM Sep 6, 2023
ded9cfa
update doc
ToniRamirezM Sep 6, 2023
fa7a79a
add rollback and entries definition
ToniRamirezM Sep 7, 2023
f41cbc2
update to alpha3
ToniRamirezM Sep 7, 2023
89a3847
fix missing dbTx
ToniRamirezM Sep 8, 2023
79d02a0
update to alpha4
ToniRamirezM Sep 8, 2023
e449f10
update to alpha4
ToniRamirezM Sep 8, 2023
13e0301
Fix rom ooc error detection (#2527) (#2528)
ARR552 Sep 8, 2023
0b021af
readme change
ARR552 Sep 8, 2023
d66fba1
cherry-pick #2524 from develop into v0.3.1 (#2531)
agnusmor Sep 9, 2023
e3063d5
improve: adding handling of odd-length hex strings in .
Psykepro Aug 31, 2023
fe421a1
Merge pull request #2533 from 0xPolygonHermez/cherry-pick/decode-hex-…
Psykepro Sep 11, 2023
a6cd85f
Merge branch 'release/v0.3.1' into feature/sequencerDataStream
ToniRamirezM Sep 12, 2023
c499058
refactor
ToniRamirezM Sep 12, 2023
9db47c1
update doc
ToniRamirezM Sep 12, 2023
d74aee4
fix test
ToniRamirezM Sep 12, 2023
adbe5c4
fix test
ToniRamirezM Sep 12, 2023
c62e9dc
fix config test
ToniRamirezM Sep 12, 2023
5f6e28f
fix config test
ToniRamirezM Sep 12, 2023
315a2cc
fix config test
ToniRamirezM Sep 12, 2023
20beebd
fix config test
ToniRamirezM Sep 12, 2023
e617ef0
fix synch test
ToniRamirezM Sep 12, 2023
1de3137
fix synch test
ToniRamirezM Sep 12, 2023
3c9f38e
fix synch test
ToniRamirezM Sep 12, 2023
0bace42
fix synch test
ToniRamirezM Sep 12, 2023
0c05d3a
fix synch test
ToniRamirezM Sep 12, 2023
1b3f9cf
fix synch test
ToniRamirezM Sep 12, 2023
7a9d134
comments
ToniRamirezM Sep 14, 2023
cab4cec
udpate data stream version
ToniRamirezM Sep 14, 2023
413e9e5
revert geth version
ToniRamirezM Sep 14, 2023
856f97f
udpate geth version
ToniRamirezM Sep 14, 2023
ae5fa60
fix trace test
dependabot[bot] Sep 15, 2023
1926eaf
update data streamer module version
ToniRamirezM Sep 19, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions config/config_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -172,6 +172,18 @@ func Test_Defaults(t *testing.T) {
path: "Sequencer.DBManager.L2ReorgRetrievalInterval",
expectedValue: types.NewDuration(5 * time.Second),
},
{
path: "Sequencer.StreamServer.Port",
expectedValue: uint16(0),
},
{
path: "Sequencer.StreamServer.Filename",
expectedValue: "",
},
{
path: "Sequencer.StreamServer.Enabled",
expectedValue: false,
},
{
path: "SequenceSender.WaitPeriodSendSequence",
expectedValue: types.NewDuration(5 * time.Second),
Expand Down
4 changes: 4 additions & 0 deletions config/default.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -110,6 +110,10 @@ MaxTxLifetime = "3h"
ByteGasCost = 16
MarginFactor = 1
Enabled = false
[Sequencer.StreamServer]
ToniRamirezM marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

StreamServer or DataStreamer? As we have used DataStreamerCfg for the config struct

Port = 0
Filename = ""
Enabled = false

[SequenceSender]
WaitPeriodSendSequence = "5s"
Expand Down
4 changes: 4 additions & 0 deletions config/environments/local/local.node.config.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,10 @@ MaxTxLifetime = "3h"
ByteGasCost = 16
MarginFactor = 1
Enabled = false
[Sequencer.StreamServer]
Port = 0
Psykepro marked this conversation as resolved.
Show resolved Hide resolved
Filename = ""
Enabled = false

[SequenceSender]
WaitPeriodSendSequence = "5s"
Expand Down
8 changes: 4 additions & 4 deletions docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ services:
environment:
- ZKEVM_NODE_ETHERMAN_URL=${ZKEVM_NODE_ETHERMAN_URL}
volumes:
- ${ZKEVM_ADVANCED_CONFIG_DIR:-./config/environments/testnet}/node.config.toml:/app/config.toml
- ${ZKEVM_ADVANCED_CONFIG_DIR:-./config/environments/${ZKEVM_NETWORK}}/node.config.toml:/app/config.toml
command:
- "/bin/sh"
- "-c"
Expand All @@ -50,7 +50,7 @@ services:
environment:
- ZKEVM_NODE_ETHERMAN_URL=${ZKEVM_NODE_ETHERMAN_URL}
volumes:
- ${ZKEVM_ADVANCED_CONFIG_DIR:-./config/environments/testnet}/node.config.toml:/app/config.toml
- ${ZKEVM_ADVANCED_CONFIG_DIR:-./config/environments/${ZKEVM_NETWORK}}/node.config.toml:/app/config.toml
command:
- "/bin/sh"
- "-c"
Expand All @@ -70,7 +70,7 @@ services:
volumes:
- ./db/scripts/init_prover_db.sql:/docker-entrypoint-initdb.d/init.sql
- ${ZKEVM_NODE_STATEDB_DATA_DIR}:/var/lib/postgresql/data
- ${ZKEVM_ADVANCED_CONFIG_DIR:-./config/environments/testnet}/postgresql.conf:/etc/postgresql.conf
- ${ZKEVM_ADVANCED_CONFIG_DIR:-./config/environments/${ZKEVM_NETWORK}}/postgresql.conf:/etc/postgresql.conf
environment:
- POSTGRES_USER=state_user
- POSTGRES_PASSWORD=state_password
Expand Down Expand Up @@ -115,6 +115,6 @@ services:
- 50061:50061 # MT
- 50071:50071 # Executor
volumes:
- ${ZKEVM_ADVANCED_CONFIG_DIR:-./config/environments/testnet}/prover.config.json:/usr/src/app/config.json
- ${ZKEVM_ADVANCED_CONFIG_DIR:-./config/environments/${ZKEVM_NETWORK}}/prover.config.json:/usr/src/app/config.json
command: >
zkProver -c /usr/src/app/config.json
2 changes: 1 addition & 1 deletion docs/config-file/node-config-doc.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@
</pre></div> </div><div id=Sequencer_DBManager_PoolRetrievalInterval_ex2 class="jumbotron examples"><div class=highlight><pre><span></span><span class=s2>&quot;300ms&quot;</span>
</pre></div> </div> <hr> <div class=breadcrumbs> <!-- None --><!-- None --><!-- None --><a href=#Sequencer.DBManager.L2ReorgRetrievalInterval onclick="anchorLink('Sequencer.DBManager.L2ReorgRetrievalInterval')">Sequencer.DBManager.L2ReorgRetrievalInterval=</a> </div> <span class="badge badge-success default-value">Default: "5s"</span><span class="badge badge-dark value-type">Type: string</span><br> <br> <div class="badge badge-secondary">Examples:</div> <br><div id=Sequencer_DBManager_L2ReorgRetrievalInterval_ex1 class="jumbotron examples"><div class=highlight><pre><span></span><span class=s2>&quot;1m&quot;</span>
</pre></div> </div><div id=Sequencer_DBManager_L2ReorgRetrievalInterval_ex2 class="jumbotron examples"><div class=highlight><pre><span></span><span class=s2>&quot;300ms&quot;</span>
</pre></div> </div> <hr> </div> </div> </div> </div> <div class=accordion id=accordionSequencer_EffectiveGasPrice> <div class=card> <div class=card-header id=headingSequencer_EffectiveGasPrice> <h2 class=mb-0> <button class="btn btn-link property-name-button" type=button data-toggle=collapse data-target=#Sequencer_EffectiveGasPrice aria-expanded aria-controls=Sequencer_EffectiveGasPrice onclick="setAnchor('#Sequencer_EffectiveGasPrice')"><span class=property-name> <div class=breadcrumbs>[<a href=#Sequencer onclick="anchorLink('Sequencer')">Sequencer</a> . <a href=#Sequencer_EffectiveGasPrice onclick="anchorLink('Sequencer_EffectiveGasPrice')">EffectiveGasPrice</a>] </div></span></button> </h2> EffectiveGasPrice is the config for the gas price </div> <div id=Sequencer_EffectiveGasPrice class="collapse property-definition-div" aria-labelledby=headingSequencer_EffectiveGasPrice data-parent=#accordionSequencer_EffectiveGasPrice> <div class="card-body pl-5"> <div class=breadcrumbs> <!-- None --><!-- None --><!-- None --><a href=#Sequencer.EffectiveGasPrice.MaxBreakEvenGasPriceDeviationPercentage onclick="anchorLink('Sequencer.EffectiveGasPrice.MaxBreakEvenGasPriceDeviationPercentage')">Sequencer.EffectiveGasPrice.MaxBreakEvenGasPriceDeviationPercentage=</a> </div> <span class="badge badge-success default-value">Default: 10</span><span class="badge badge-dark value-type">Type: integer</span><br> <span class=description><p>MaxBreakEvenGasPriceDeviationPercentage is the max allowed deviation percentage BreakEvenGasPrice on re-calculation</p> </span> <hr> <div class=breadcrumbs> <!-- None --><!-- None --><!-- None --><a href=#Sequencer.EffectiveGasPrice.L1GasPriceFactor onclick="anchorLink('Sequencer.EffectiveGasPrice.L1GasPriceFactor')">Sequencer.EffectiveGasPrice.L1GasPriceFactor=</a> </div> <span class="badge badge-success default-value">Default: 0.25</span><span class="badge badge-dark value-type">Type: number</span><br> <span class=description><p>L1GasPriceFactor is the percentage of the L1 gas price that will be used as the L2 min gas price</p> </span> <hr> <div class=breadcrumbs> <!-- None --><!-- None --><!-- None --><a href=#Sequencer.EffectiveGasPrice.ByteGasCost onclick="anchorLink('Sequencer.EffectiveGasPrice.ByteGasCost')">Sequencer.EffectiveGasPrice.ByteGasCost=</a> </div> <span class="badge badge-success default-value">Default: 16</span><span class="badge badge-dark value-type">Type: integer</span><br> <span class=description><p>ByteGasCost is the gas cost per byte</p> </span> <hr> <div class=breadcrumbs> <!-- None --><!-- None --><!-- None --><a href=#Sequencer.EffectiveGasPrice.MarginFactor onclick="anchorLink('Sequencer.EffectiveGasPrice.MarginFactor')">Sequencer.EffectiveGasPrice.MarginFactor=</a> </div> <span class="badge badge-success default-value">Default: 1</span><span class="badge badge-dark value-type">Type: number</span><br> <span class=description><p>MarginFactor is the margin factor percentage to be added to the L2 min gas price</p> </span> <hr> <div class=breadcrumbs> <!-- None --><!-- None --><!-- None --><a href=#Sequencer.EffectiveGasPrice.Enabled onclick="anchorLink('Sequencer.EffectiveGasPrice.Enabled')">Sequencer.EffectiveGasPrice.Enabled=</a> </div> <span class="badge badge-success default-value">Default: false</span><span class="badge badge-dark value-type">Type: boolean</span><br> <span class=description><p>Enabled is a flag to enable/disable the effective gas price</p> </span> <hr> <div class=breadcrumbs> <!-- None --><!-- None --><!-- None --><a href=#Sequencer.EffectiveGasPrice.DefaultMinGasPriceAllowed onclick="anchorLink('Sequencer.EffectiveGasPrice.DefaultMinGasPriceAllowed')">Sequencer.EffectiveGasPrice.DefaultMinGasPriceAllowed=</a> </div> <span class="badge badge-success default-value">Default: 0</span><span class="badge badge-dark value-type">Type: integer</span><br> <span class=description><p>DefaultMinGasPriceAllowed is the default min gas price to suggest<br> This value is assigned from [Pool].DefaultMinGasPriceAllowed</p> </span> <hr> </div> </div> </div> </div> </div> </div> </div> </div> <div class=accordion id=accordionSequenceSender> <div class=card> <div class=card-header id=headingSequenceSender> <h2 class=mb-0> <button class="btn btn-link property-name-button" type=button data-toggle=collapse data-target=#SequenceSender aria-expanded aria-controls=SequenceSender onclick="setAnchor('#SequenceSender')"><span class=property-name> <div class=breadcrumbs>[<a href=#SequenceSender onclick="anchorLink('SequenceSender')">SequenceSender</a>] </div></span></button> </h2> Configuration of the sequence sender service </div> <div id=SequenceSender class="collapse property-definition-div" aria-labelledby=headingSequenceSender data-parent=#accordionSequenceSender> <div class="card-body pl-5"> <div class=breadcrumbs> <!-- None --><!-- None --><a href=#SequenceSender.WaitPeriodSendSequence onclick="anchorLink('SequenceSender.WaitPeriodSendSequence')">SequenceSender.WaitPeriodSendSequence=</a> </div> <span class="badge badge-success default-value">Default: "5s"</span><span class="badge badge-dark value-type">Type: string</span><br> <span class=description><p>WaitPeriodSendSequence is the time the sequencer waits until<br> trying to send a sequence to L1</p> </span> <br> <div class="badge badge-secondary">Examples:</div> <br><div id=SequenceSender_WaitPeriodSendSequence_ex1 class="jumbotron examples"><div class=highlight><pre><span></span><span class=s2>&quot;1m&quot;</span>
</pre></div> </div> <hr> </div> </div> </div> </div> <div class=accordion id=accordionSequencer_EffectiveGasPrice> <div class=card> <div class=card-header id=headingSequencer_EffectiveGasPrice> <h2 class=mb-0> <button class="btn btn-link property-name-button" type=button data-toggle=collapse data-target=#Sequencer_EffectiveGasPrice aria-expanded aria-controls=Sequencer_EffectiveGasPrice onclick="setAnchor('#Sequencer_EffectiveGasPrice')"><span class=property-name> <div class=breadcrumbs>[<a href=#Sequencer onclick="anchorLink('Sequencer')">Sequencer</a> . <a href=#Sequencer_EffectiveGasPrice onclick="anchorLink('Sequencer_EffectiveGasPrice')">EffectiveGasPrice</a>] </div></span></button> </h2> EffectiveGasPrice is the config for the gas price </div> <div id=Sequencer_EffectiveGasPrice class="collapse property-definition-div" aria-labelledby=headingSequencer_EffectiveGasPrice data-parent=#accordionSequencer_EffectiveGasPrice> <div class="card-body pl-5"> <div class=breadcrumbs> <!-- None --><!-- None --><!-- None --><a href=#Sequencer.EffectiveGasPrice.MaxBreakEvenGasPriceDeviationPercentage onclick="anchorLink('Sequencer.EffectiveGasPrice.MaxBreakEvenGasPriceDeviationPercentage')">Sequencer.EffectiveGasPrice.MaxBreakEvenGasPriceDeviationPercentage=</a> </div> <span class="badge badge-success default-value">Default: 10</span><span class="badge badge-dark value-type">Type: integer</span><br> <span class=description><p>MaxBreakEvenGasPriceDeviationPercentage is the max allowed deviation percentage BreakEvenGasPrice on re-calculation</p> </span> <hr> <div class=breadcrumbs> <!-- None --><!-- None --><!-- None --><a href=#Sequencer.EffectiveGasPrice.L1GasPriceFactor onclick="anchorLink('Sequencer.EffectiveGasPrice.L1GasPriceFactor')">Sequencer.EffectiveGasPrice.L1GasPriceFactor=</a> </div> <span class="badge badge-success default-value">Default: 0.25</span><span class="badge badge-dark value-type">Type: number</span><br> <span class=description><p>L1GasPriceFactor is the percentage of the L1 gas price that will be used as the L2 min gas price</p> </span> <hr> <div class=breadcrumbs> <!-- None --><!-- None --><!-- None --><a href=#Sequencer.EffectiveGasPrice.ByteGasCost onclick="anchorLink('Sequencer.EffectiveGasPrice.ByteGasCost')">Sequencer.EffectiveGasPrice.ByteGasCost=</a> </div> <span class="badge badge-success default-value">Default: 16</span><span class="badge badge-dark value-type">Type: integer</span><br> <span class=description><p>ByteGasCost is the gas cost per byte</p> </span> <hr> <div class=breadcrumbs> <!-- None --><!-- None --><!-- None --><a href=#Sequencer.EffectiveGasPrice.MarginFactor onclick="anchorLink('Sequencer.EffectiveGasPrice.MarginFactor')">Sequencer.EffectiveGasPrice.MarginFactor=</a> </div> <span class="badge badge-success default-value">Default: 1</span><span class="badge badge-dark value-type">Type: number</span><br> <span class=description><p>MarginFactor is the margin factor percentage to be added to the L2 min gas price</p> </span> <hr> <div class=breadcrumbs> <!-- None --><!-- None --><!-- None --><a href=#Sequencer.EffectiveGasPrice.Enabled onclick="anchorLink('Sequencer.EffectiveGasPrice.Enabled')">Sequencer.EffectiveGasPrice.Enabled=</a> </div> <span class="badge badge-success default-value">Default: false</span><span class="badge badge-dark value-type">Type: boolean</span><br> <span class=description><p>Enabled is a flag to enable/disable the effective gas price</p> </span> <hr> <div class=breadcrumbs> <!-- None --><!-- None --><!-- None --><a href=#Sequencer.EffectiveGasPrice.DefaultMinGasPriceAllowed onclick="anchorLink('Sequencer.EffectiveGasPrice.DefaultMinGasPriceAllowed')">Sequencer.EffectiveGasPrice.DefaultMinGasPriceAllowed=</a> </div> <span class="badge badge-success default-value">Default: 0</span><span class="badge badge-dark value-type">Type: integer</span><br> <span class=description><p>DefaultMinGasPriceAllowed is the default min gas price to suggest<br> This value is assigned from [Pool].DefaultMinGasPriceAllowed</p> </span> <hr> </div> </div> </div> </div> <div class=accordion id=accordionSequencer_StreamServer> <div class=card> <div class=card-header id=headingSequencer_StreamServer> <h2 class=mb-0> <button class="btn btn-link property-name-button" type=button data-toggle=collapse data-target=#Sequencer_StreamServer aria-expanded aria-controls=Sequencer_StreamServer onclick="setAnchor('#Sequencer_StreamServer')"><span class=property-name> <div class=breadcrumbs>[<a href=#Sequencer onclick="anchorLink('Sequencer')">Sequencer</a> . <a href=#Sequencer_StreamServer onclick="anchorLink('Sequencer_StreamServer')">StreamServer</a>] </div></span></button> </h2> StreamServerCfg is the config for the stream server </div> <div id=Sequencer_StreamServer class="collapse property-definition-div" aria-labelledby=headingSequencer_StreamServer data-parent=#accordionSequencer_StreamServer> <div class="card-body pl-5"> <div class=breadcrumbs> <!-- None --><!-- None --><!-- None --><a href=#Sequencer.StreamServer.Port onclick="anchorLink('Sequencer.StreamServer.Port')">Sequencer.StreamServer.Port=</a> </div> <span class="badge badge-success default-value">Default: 0</span><span class="badge badge-dark value-type">Type: integer</span><br> <span class=description><p>Port to listen on</p> </span> <hr> <div class=breadcrumbs> <!-- None --><!-- None --><!-- None --><a href=#Sequencer.StreamServer.Filename onclick="anchorLink('Sequencer.StreamServer.Filename')">Sequencer.StreamServer.Filename=</a> </div> <span class="badge badge-success default-value">Default: ""</span><span class="badge badge-dark value-type">Type: string</span><br> <span class=description><p>Filename of the binary data file</p> </span> <hr> <div class=breadcrumbs> <!-- None --><!-- None --><!-- None --><a href=#Sequencer.StreamServer.Enabled onclick="anchorLink('Sequencer.StreamServer.Enabled')">Sequencer.StreamServer.Enabled=</a> </div> <span class="badge badge-success default-value">Default: false</span><span class="badge badge-dark value-type">Type: boolean</span><br> <span class=description><p>Enabled is a flag to enable/disable the data streamer</p> </span> <hr> </div> </div> </div> </div> </div> </div> </div> </div> <div class=accordion id=accordionSequenceSender> <div class=card> <div class=card-header id=headingSequenceSender> <h2 class=mb-0> <button class="btn btn-link property-name-button" type=button data-toggle=collapse data-target=#SequenceSender aria-expanded aria-controls=SequenceSender onclick="setAnchor('#SequenceSender')"><span class=property-name> <div class=breadcrumbs>[<a href=#SequenceSender onclick="anchorLink('SequenceSender')">SequenceSender</a>] </div></span></button> </h2> Configuration of the sequence sender service </div> <div id=SequenceSender class="collapse property-definition-div" aria-labelledby=headingSequenceSender data-parent=#accordionSequenceSender> <div class="card-body pl-5"> <div class=breadcrumbs> <!-- None --><!-- None --><a href=#SequenceSender.WaitPeriodSendSequence onclick="anchorLink('SequenceSender.WaitPeriodSendSequence')">SequenceSender.WaitPeriodSendSequence=</a> </div> <span class="badge badge-success default-value">Default: "5s"</span><span class="badge badge-dark value-type">Type: string</span><br> <span class=description><p>WaitPeriodSendSequence is the time the sequencer waits until<br> trying to send a sequence to L1</p> </span> <br> <div class="badge badge-secondary">Examples:</div> <br><div id=SequenceSender_WaitPeriodSendSequence_ex1 class="jumbotron examples"><div class=highlight><pre><span></span><span class=s2>&quot;1m&quot;</span>
</pre></div> </div><div id=SequenceSender_WaitPeriodSendSequence_ex2 class="jumbotron examples"><div class=highlight><pre><span></span><span class=s2>&quot;300ms&quot;</span>
</pre></div> </div> <hr> <div class=breadcrumbs> <!-- None --><!-- None --><a href=#SequenceSender.LastBatchVirtualizationTimeMaxWaitPeriod onclick="anchorLink('SequenceSender.LastBatchVirtualizationTimeMaxWaitPeriod')">SequenceSender.LastBatchVirtualizationTimeMaxWaitPeriod=</a> </div> <span class="badge badge-success default-value">Default: "5s"</span><span class="badge badge-dark value-type">Type: string</span><br> <span class=description><p>LastBatchVirtualizationTimeMaxWaitPeriod is time since sequences should be sent</p> </span> <br> <div class="badge badge-secondary">Examples:</div> <br><div id=SequenceSender_LastBatchVirtualizationTimeMaxWaitPeriod_ex1 class="jumbotron examples"><div class=highlight><pre><span></span><span class=s2>&quot;1m&quot;</span>
</pre></div> </div><div id=SequenceSender_LastBatchVirtualizationTimeMaxWaitPeriod_ex2 class="jumbotron examples"><div class=highlight><pre><span></span><span class=s2>&quot;300ms&quot;</span>
Expand Down
Loading