Add integration of dragon-based event broadcasting #710

ankona · 2024-09-18T15:25:46Z

This PR integrates event publishers and consumers in ProtoClient and DragonBackend

codecov · 2024-09-18T15:41:53Z

Codecov Report

Attention: Patch coverage is 2.40506% with 771 lines in your changes missing coverage. Please review.

Please upload report for BASE (mli-feature@5ec287c). Learn more about missing BASE report.

Files with missing lines	Patch %	Lines
smartsim/_core/mli/client/protoclient.py	0.00%	144 Missing ⚠️
...rtsim/_core/mli/infrastructure/control/listener.py	0.00%	131 Missing ⚠️
smartsim/_core/mli/infrastructure/comm/consumer.py	0.00%	110 Missing ⚠️
...rtsim/_core/mli/infrastructure/comm/broadcaster.py	0.00%	105 Missing ⚠️
...i/infrastructure/storage/backbone_feature_store.py	0.00%	82 Missing ⚠️
smartsim/_core/mli/infrastructure/comm/event.py	0.00%	60 Missing ⚠️
smartsim/_core/launcher/dragon/dragonBackend.py	4.54%	42 Missing ⚠️
...im/_core/mli/infrastructure/storage/dragon_util.py	0.00%	21 Missing ⚠️
.../_core/mli/infrastructure/storage/feature_store.py	0.00%	18 Missing ⚠️
...mli/infrastructure/storage/dragon_feature_store.py	0.00%	15 Missing ⚠️
... and 6 more

Additional details and impacted files

@@              Coverage Diff               @@
##             mli-feature     #710   +/-   ##
==============================================
  Coverage               ?   64.96%           
==============================================
  Files                  ?      110           
  Lines                  ?     9419           
  Branches               ?        0           
==============================================
  Hits                   ?     6119           
  Misses                 ?     3300           
  Partials               ?        0

Files with missing lines	Coverage Δ
..._core/mli/infrastructure/control/error_handling.py	`0.00% <ø> (ø)`
smartsim/_core/mli/message_handler.py	`97.81% <100.00%> (ø)`
smartsim/_core/utils/timings.py	`0.00% <ø> (ø)`
smartsim/_core/launcher/dragon/dragonConnector.py	`68.23% <91.66%> (ø)`
smartsim/log.py	`94.40% <33.33%> (ø)`
smartsim/_core/mli/infrastructure/comm/producer.py	`0.00% <0.00%> (ø)`
...sim/_core/mli/infrastructure/environment_loader.py	`0.00% <0.00%> (ø)`
smartsim/_core/mli/infrastructure/worker/worker.py	`0.00% <0.00%> (ø)`
...e/mli/infrastructure/control/request_dispatcher.py	`0.00% <0.00%> (ø)`
...mli/infrastructure/storage/dragon_feature_store.py	`0.00% <0.00%> (ø)`
... and 9 more

Revert tensor/model key, update tests, enhance logging Tweak exception naming to follow standard Test remote queue delay and nowait Remove large test timeout Update new tests w/fsk reversion Modify import order for remote tester

…agon utils module, fix missing test env vars, ...

… reuse some test fixtures for speed

…measurement error

…ted code

… use fixtures for pytest ddict creation

mellis13

Really great additions that get us closer to the online notifications. Mostly minor questions/comments

smartsim/_core/mli/comm/channel/dragon_util.py

smartsim/_core/mli/comm/channel/dragon_channel.py

smartsim/_core/mli/comm/channel/dragon_util.py

smartsim/_core/mli/comm/channel/dragon_channel.py

smartsim/_core/mli/comm/channel/dragon_fli.py

smartsim/_core/mli/infrastructure/storage/backbone_feature_store.py

al-rigazzi

A few comments, but overall looks impressive

conftest.py

ex/high_throughput_inference/mock_app.py

smartsim/_core/mli/comm/channel/dragon_util.py

smartsim/_core/launcher/dragon/dragonBackend.py

smartsim/_core/mli/comm/channel/dragon_channel.py

smartsim/_core/mli/comm/channel/dragon_fli.py

smartsim/_core/_cli/scripts/dragon_install.py

smartsim/_core/entrypoints/service.py

smartsim/_core/mli/infrastructure/control/request_dispatcher.py

AlyssaCote

The other reviewers had already caught any of the questions or suggestions I had. This is really well done!

smartsim/protoclient.py

… consumers, moar tests

amandarichardsonn

I must commend thee on thy splendid work! I have conjured a few suggestions that might enhance your creation, but feel free to disregard any that do not align with your vision. I have perused about halfway through and shall return to complete my review with haste.

smartsim/_core/launcher/dragon/dragonBackend.py

smartsim/_core/mli/comm/channel/channel.py

smartsim/_core/mli/comm/channel/dragon_fli.py

smartsim/_core/mli/infrastructure/control/event_listener.py

amandarichardsonn

Some more comments, slowly looking through

smartsim/_core/mli/infrastructure/storage/backbone_feature_store.py

smartsim/protoclient.py

smartsim/_core/mli/infrastructure/worker/worker.py

amandarichardsonn

Partial review, stopping at consumer.py

smartsim/_core/entrypoints/service.py

smartsim/_core/launcher/dragon/dragonBackend.py

smartsim/_core/launcher/dragon/dragonConnector.py

smartsim/_core/mli/infrastructure/comm/broadcaster.py

smartsim/_core/mli/infrastructure/comm/consumer.py

al-rigazzi

Only a few nitpicks otherwise looks great!

conftest.py

smartsim/_core/entrypoints/service.py

smartsim/_core/launcher/dragon/dragonBackend.py

smartsim/_core/mli/client/protoclient.py

smartsim/_core/mli/comm/channel/dragon_channel.py

smartsim/_core/mli/infrastructure/comm/broadcaster.py

smartsim/_core/mli/infrastructure/comm/consumer.py

smartsim/_core/mli/infrastructure/comm/broadcaster.py

amandarichardsonn

Reviewed rest of files excluding /tests, incredible work!

smartsim/_core/mli/infrastructure/control/listener.py

smartsim/_core/mli/infrastructure/control/request_dispatcher.py

smartsim/_core/mli/infrastructure/worker/worker.py

smartsim/_core/mli/message_handler.py

al-rigazzi

LGTM!

ankona force-pushed the 747-int-4 branch from cd23db4 to 4b6ade0 Compare September 18, 2024 15:35

ankona force-pushed the 747-int-4 branch from 4b6ade0 to 8ee2faa Compare September 18, 2024 15:43

Squash event integration

c6a145a

ankona force-pushed the 747-int-4 branch from 91cca53 to 29d1fe0 Compare September 25, 2024 15:38

ankona and others added 10 commits September 25, 2024 18:33

Revert featurestorekey test changes

49e0da4

Revert tensor/model key, update tests, enhance logging Tweak exception naming to follow standard Test remote queue delay and nowait Remove large test timeout Update new tests w/fsk reversion Modify import order for remote tester

capture dragon start method ex

fd0a5ec

Split message pump into separate module from dispatcher test

00a4496

extract msg_pump_factory for reuse in test_worker_manager.py

a87aba7

add test verifying protoclient event raises

4879122

reduce timeouts & backoffs, share backbone across protoclient tests

6f1cba7

Test eventing end-to-end in single process

c0a1bca

docstrings & miscellaneous minor fixes (reuse descriptor code, add dr…

d81334d

…agon utils module, fix missing test env vars, ...

fix infinite loop bug in consumer batch receive

f5b7b7d

Sort imports to solve dragon import issue in non-dragon tests

bd03711

ankona force-pushed the 747-int-4 branch from 95c10b3 to bd03711 Compare September 25, 2024 23:34

swap session scopes to module to destroy dragon resources

98260f3

ankona changed the title ~~747 int 4~~ Add integration of dragon-based event broadcasting Sep 26, 2024

ankona requested review from AlyssaCote and al-rigazzi and removed request for AlyssaCote September 26, 2024 15:58

ankona added 8 commits September 26, 2024 12:46

use make process local to avoid MPI issue, fix some test regressions,…

5ba2a42

… reuse some test fixtures for speed

more docstrings standard fixes

1f4e6e3

reduce default worker connect timeout, fix test timeout issue due to …

9c8d127

…measurement error

use constants in tests for env var strings, docstrings, remove commen…

7442eb1

…ted code

docstring formatting in tests

da20b5f

docstrings

f5ba5a6

parameterize ddict creation, add single ddict touchpoint util module,…

d689754

… use fixtures for pytest ddict creation

remove completed todos, fix docstrings, remove obsolete/commented code

7ddcd7c

mellis13 suggested changes Sep 30, 2024

View reviewed changes

al-rigazzi requested changes Oct 1, 2024

View reviewed changes

AlyssaCote reviewed Oct 1, 2024

View reviewed changes

smartsim/protoclient.py Outdated Show resolved Hide resolved

ankona added 2 commits October 1, 2024 18:02

review changes part 1, improve dragon errors handling, add unregister…

979373c

… consumers, moar tests

import order follow-up

5898005

amandarichardsonn reviewed Oct 2, 2024

View reviewed changes

amandarichardsonn reviewed Oct 3, 2024

View reviewed changes

ankona added 3 commits October 3, 2024 19:41

pr review updates

af870f9

fix extra arg copypasta

27ab4f9

test bugs

fac30bd

ankona requested a review from al-rigazzi October 4, 2024 20:45

ankona added 2 commits October 7, 2024 14:02

Remove from_sender_supplied_descriptor factory on FLI channel

a1cf7ff

re-home protoclient into mli subpackage

124a195

amandarichardsonn reviewed Oct 7, 2024

View reviewed changes

al-rigazzi requested changes Oct 7, 2024

View reviewed changes

additional missed docstrings

b2c4cb7

amandarichardsonn reviewed Oct 8, 2024

View reviewed changes

ankona added 6 commits October 8, 2024 11:46

improve service param docstrings, avoid separate var/param descriptions

0c495f6

docstring fixes

1fc59e4

increase default memory per backbone node

b316128

fix fixture usage bug (worker queue preloaded into backbone)

2ed47a4

extract dragon fixtures into dragon conftest.py

8ccebb5

remove unused import

608d6bd

ankona force-pushed the 747-int-4 branch from 170c9ea to 608d6bd Compare October 9, 2024 00:50

fix send-multi with FLI after sender-supplied channel removal

68d0d0c

al-rigazzi approved these changes Oct 9, 2024

View reviewed changes

ankona added 2 commits October 9, 2024 16:44

Update dispatch tests to use dragon processes

c28870f

Use cached FLI channel on single-send

78d5598

ankona requested a review from al-rigazzi October 9, 2024 22:12

al-rigazzi approved these changes Oct 9, 2024

View reviewed changes

ankona merged commit ca01cb1 into CrayLabs:mli-feature Oct 10, 2024
79 of 85 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add integration of dragon-based event broadcasting #710

Add integration of dragon-based event broadcasting #710

ankona commented Sep 18, 2024 •

edited

Loading

codecov bot commented Sep 18, 2024 •

edited

Loading

mellis13 left a comment

al-rigazzi left a comment

AlyssaCote left a comment

amandarichardsonn left a comment

amandarichardsonn left a comment

amandarichardsonn left a comment

al-rigazzi left a comment

amandarichardsonn left a comment

al-rigazzi left a comment

Add integration of dragon-based event broadcasting #710

Add integration of dragon-based event broadcasting #710

Conversation

ankona commented Sep 18, 2024 • edited Loading

codecov bot commented Sep 18, 2024 • edited Loading

Codecov Report

mellis13 left a comment

Choose a reason for hiding this comment

al-rigazzi left a comment

Choose a reason for hiding this comment

AlyssaCote left a comment

Choose a reason for hiding this comment

amandarichardsonn left a comment

Choose a reason for hiding this comment

amandarichardsonn left a comment

Choose a reason for hiding this comment

amandarichardsonn left a comment

Choose a reason for hiding this comment

al-rigazzi left a comment

Choose a reason for hiding this comment

amandarichardsonn left a comment

Choose a reason for hiding this comment

al-rigazzi left a comment

Choose a reason for hiding this comment

ankona commented Sep 18, 2024 •

edited

Loading

codecov bot commented Sep 18, 2024 •

edited

Loading