Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Benph/bootstrap thread migration #3554

Merged
merged 101 commits into from
Feb 27, 2023
Merged
Show file tree
Hide file tree
Changes from 76 commits
Commits
Show all changes
101 commits
Select commit Hold shift + click to select a range
817edd6
Put in some prelimenary readability changes
Ben-PH Feb 10, 2023
a50d975
Move `bootstap_sessions.push(async move { ...} );` into helper
Ben-PH Feb 10, 2023
e5a7238
Move the bootstrtap client check into a helper method
Ben-PH Feb 10, 2023
4fb31ae
Comment BootstrapManager termination
Ben-PH Feb 13, 2023
1439558
Revert me: comment out mock establisher imports
Ben-PH Feb 13, 2023
9ab2891
Use `BootstrapServerBinder`
Ben-PH Feb 13, 2023
e7b63e0
Spawn bootstrap management on a seperate tokio::spawn thread
Ben-PH Feb 14, 2023
e909730
Minor cleanup/comment
Ben-PH Feb 14, 2023
95f00f8
Revert multi-threaded active-sessions counter
Ben-PH Feb 14, 2023
e81c57b
Merge remote-tracking branch 'origin/testnet_20' into benph/bootstrap…
Ben-PH Feb 14, 2023
64dd623
use `Arc<()>` strong-count to track bootstrap session count
Ben-PH Feb 14, 2023
82e9fa7
use let-else-continue
Ben-PH Feb 14, 2023
c5f227a
Update doc-comment
Ben-PH Feb 14, 2023
45d59d2
Refine the use of bootstrap-session-counting
Ben-PH Feb 14, 2023
e5fe0ea
Fix arc-count obo error
Ben-PH Feb 15, 2023
9f54ebf
Self-review
Ben-PH Feb 15, 2023
703913e
Refine some (doc-)comments
Ben-PH Feb 15, 2023
f547c0c
Revert the channel usage. Log completion within the handler directly
Ben-PH Feb 15, 2023
fb6c954
revert-me: comment out test type-decs in establisher for dev-workflow
Ben-PH Feb 16, 2023
c4bfa6b
Initial sketch of how things will be less async-y
Ben-PH Feb 16, 2023
18ba93d
Have the allow-block list stuff handled by dedicated helper-module
Ben-PH Feb 16, 2023
ec17622
Cleanup code-style of allow/block-list module
Ben-PH Feb 16, 2023
6189174
Some comments/tidy-up to pave the way forward
Ben-PH Feb 16, 2023
9d674a9
Clean up the client-interval check
Ben-PH Feb 16, 2023
7229aa9
Delete commented out code that was migrated to lists helper module
Ben-PH Feb 16, 2023
b5bff46
Address some WIP todos
Ben-PH Feb 16, 2023
7303255
Put the main worker-loop into a dedicated helper method
Ben-PH Feb 16, 2023
2defb75
Handle update error, and don't use over-sized enum
Ben-PH Feb 16, 2023
1208214
Restore compile and clippy warnings
Ben-PH Feb 16, 2023
56bb256
Broken: One of the async contexts doesn't run. Add todo! to mark loca…
Ben-PH Feb 17, 2023
12270f4
Merge remote-tracking branch 'origin/testnet_20' into benph/bootstrap…
Ben-PH Feb 17, 2023
8b48ff0
Use helper method to send errors in a timeout
Ben-PH Feb 17, 2023
3965ded
First naive attempt at taking out some async
Ben-PH Feb 17, 2023
92af76b
Has problems with running the bootstrap session runner
Ben-PH Feb 17, 2023
028e15d
revert me: comment out test retyping for workflow benefit
Ben-PH Feb 17, 2023
5373f8d
field/variable renames
Ben-PH Feb 17, 2023
6c3bee1
prototype the `start.await.unwrap()` with internal run
Ben-PH Feb 17, 2023
1628adb
Use substruct for serverbinding config
Ben-PH Feb 17, 2023
e78bb77
Make it start making a bit of sense
Ben-PH Feb 17, 2023
243e80e
reinstate test compiling for establisher types
Ben-PH Feb 17, 2023
7bdee2b
get tests compiling again
Ben-PH Feb 17, 2023
1ea3c1c
Pass the test
Ben-PH Feb 20, 2023
551d433
Use helper motheds to handle the message sending
Ben-PH Feb 20, 2023
8ce1384
Clear old, dead commented out code
Ben-PH Feb 20, 2023
ffc2419
Add some todo's
Ben-PH Feb 20, 2023
adf9326
Run bootstrap server in a spawned thread, with dedicated tokio runtime
Ben-PH Feb 21, 2023
94c8dab
Doc-comment the module
Ben-PH Feb 21, 2023
d451c67
Make stop non-async
Ben-PH Feb 21, 2023
b187775
revert back to channels
Ben-PH Feb 21, 2023
d5bb079
comments and selector use
Ben-PH Feb 21, 2023
11b7850
refine select use, and get the test passing
Ben-PH Feb 21, 2023
d25fdb7
Merge branch 'benph-lillian/bootstrap-deasync' into benph/bootstrap-d…
Ben-PH Feb 21, 2023
ac3037a
Clarify the use of the Select type through comments/readability refactor
Ben-PH Feb 21, 2023
86b0765
Merge bootstrap-de-async work into benph/bootstrap-thread-migration
Ben-PH Feb 22, 2023
fe00ea6
Fix error in merging in work from other branch
Ben-PH Feb 22, 2023
85086a5
Refine the receiving on the listening channel
Ben-PH Feb 22, 2023
abba618
Fix errors picked up by CI
Ben-PH Feb 22, 2023
0de926a
Make the paths to allow/block lists Cow-wrapped
Ben-PH Feb 22, 2023
0bc10b7
Handle channel-error in the connection listener
Ben-PH Feb 22, 2023
0fc4d19
Move event selection to helper method
Ben-PH Feb 22, 2023
57e1c99
Clarify TODO
Ben-PH Feb 22, 2023
41d8267
Update run-bootstrap doc-comment
Ben-PH Feb 22, 2023
89f1102
Name a bootstrap thread per-address, and update debug!
Ben-PH Feb 22, 2023
2b38d2e
Refine finished bootstrap trace message
Ben-PH Feb 22, 2023
20f9387
Doc-comment the white/black list updater
Ben-PH Feb 22, 2023
26fae9a
Give `is_ip_allowed` special test behaviour
Ben-PH Feb 22, 2023
ba465d3
Clear clippy lints
Ben-PH Feb 22, 2023
49b611e
Ignore doc-comments. Correct allow-lint
Ben-PH Feb 22, 2023
dff5ab2
Claim bootstrap slot at moment of handler spawnning
Ben-PH Feb 22, 2023
da2d905
allow/block list -> white/black list
Ben-PH Feb 22, 2023
041117b
Propogate ip file-read error
Ben-PH Feb 22, 2023
cbea265
Use a close_and_send_error helper method that runs on dedicated thread
Ben-PH Feb 22, 2023
e68b296
Keep hold of the updater and listener runtime with manager
Ben-PH Feb 22, 2023
d6a946d
Fix dropping of runtime in non-blocking context
Ben-PH Feb 22, 2023
4b0f1b4
Use the servers runtime to manage async in spawned threads
Ben-PH Feb 22, 2023
0f0b1cb
Document `close_and_send_error`
Ben-PH Feb 22, 2023
956775a
Refine the thread-model
Ben-PH Feb 23, 2023
de7e959
Don't create a runtime outside of an async situation
Ben-PH Feb 23, 2023
fd813ba
Close the threads with slightly less lacking-of-grace
Ben-PH Feb 23, 2023
570c46d
Clear clippy lint
Ben-PH Feb 24, 2023
5bbe3a2
Fix missing `.await`
Ben-PH Feb 24, 2023
b991250
Make runtime dedicated to the loop
Ben-PH Feb 24, 2023
d93cd21
Improve doc-comment for listen-chan-recv
Ben-PH Feb 24, 2023
0df2f0e
Name the runtimes, and set some values with the runtime builder
Ben-PH Feb 24, 2023
07e1c32
Address/refine TODOs/comments
Ben-PH Feb 24, 2023
ec790ef
Add error for white_black_list.rs
Feb 23, 2023
d8b09a8
Clear clippy lints
Ben-PH Feb 24, 2023
7938385
Make the updater non-async
Ben-PH Feb 24, 2023
0619288
Update TODO comment
Ben-PH Feb 24, 2023
d149020
Double the connection channel capacity
Ben-PH Feb 24, 2023
13f02f5
Clean up the stop method
Ben-PH Feb 24, 2023
15c59ae
Delete dead commented-out code
Ben-PH Feb 24, 2023
3f5a9fb
Give bootstrap sessions 20s to shutdown once the main-loop is broken
Ben-PH Feb 24, 2023
6b31d4e
Include a missing `cargo fmt`
Ben-PH Feb 24, 2023
0fcd32b
Name the runtime handle that the bootstrap session uses correctly
Ben-PH Feb 24, 2023
17bfb37
Name the thread spawned by close_send_error
Ben-PH Feb 24, 2023
bf45556
Shutdown the correct runtime when out of the main-work-loop
Ben-PH Feb 24, 2023
befc93e
Apply suggestions from self code review
Ben-PH Feb 24, 2023
33c32b4
Remove debugs
Ben-PH Feb 24, 2023
ea3a6b6
Clear a TODO
Ben-PH Feb 24, 2023
09aeb5c
Add comments explaining the use of `unwrap` with thread-builder spawn
Ben-PH Feb 25, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 25 additions & 1 deletion Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

15 changes: 3 additions & 12 deletions massa-bootstrap/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,6 @@ async-speed-limit = { git = "https://github.com/adrien-zinger/async-speed-limit"
"tokio",
] }
displaydoc = "0.2"
futures = "0.3"
num_enum = "0.5"
nom = "7.1"
rand = "0.8"
Expand All @@ -39,6 +38,7 @@ massa_serialization = { path = "../massa-serialization" }
massa_signature = { path = "../massa-signature" }
massa_pos_exports = { path = "../massa-pos-exports" }
massa_time = { path = "../massa-time" }
crossbeam = "0.8.2"

[dev-dependencies]
bitvec = { version = "1.0", features = ["serde"] }
Expand All @@ -59,14 +59,5 @@ tempfile = "3.3"

# for more information on what are the following features used for, see the cargo.toml at workspace level
[features]
testing = [
"massa_final_state/testing",
"massa_ledger_worker/testing",
"massa_consensus_exports/testing",
"massa_async_pool/testing",
]
sandbox = [
"massa_async_pool/sandbox",
"massa_final_state/sandbox",
"massa_models/sandbox",
]
testing = ["massa_final_state/testing", "massa_ledger_worker/testing", "massa_consensus_exports/testing", "massa_async_pool/testing"]
sandbox = ["massa_async_pool/sandbox", "massa_final_state/sandbox", "massa_models/sandbox"]
Loading