-
Notifications
You must be signed in to change notification settings - Fork 7
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
rolling out a pipeline type system for dispatching on analysis details #224
rolling out a pipeline type system for dispatching on analysis details #224
Conversation
…tch on pipeline type
…now dispatch on pipeline type
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #224 +/- ##
=======================================
Coverage 92.73% 92.73%
=======================================
Files 42 42
Lines 399 399
=======================================
Hits 370 370
Misses 29 29 ☔ View full report in Codecov by Sentry. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The overall organisation and high level stuff is good. I don't understand the choice to keep the default functions as it appears to lead to duplication and reduced modularity with no clear upside but maybe I am missing something
Mainly, I quite like having |
…lf are passed as a Pair
I've resolved the comments around using If @seabbs could review:
That would be great. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is really nice. Only one remaining point about file paths which I think can be its own issue as not critical.
This PR implements dispatching on the type of
pipeline <: AbstractEpiAwarePipeline
to implement pipeline workflow.I have kept default constructors and in each case the pipeline-dependent constructor function just returns these. However, there is now a obvious place to extend the pipeline type tree to specify additional behaviour by writing methods for any of the component parts, e.g.
make_inference_method(pipeline)
etc.Additionally, the
AnalysisPipeline
and its unit tests have been refactored into a logical file organisation (obviously opinions make vary) and other minor fixes (e.g.rmprocs
has been added to pipeline script).NB: With reference to #221 and also following f2f meetings this PR is keeping the computation DAG as generated by
Dagger.@spawn
macros on function calls. This limits the pipeline to any backend which can interface with JuliaBase.Distributed
, and may be changed in near future.Closes #218 .