Skip to content

Commit

Permalink
coord: adjust dataflow building + shipping APIs
Browse files Browse the repository at this point in the history
Reconfiguring the APIs for building and shipping DataflowDescs results
in much simpler call sites, and hopefully easier-to-understand mutation
patterns. In the new API:

  * `build...dataflow` methods do not mutate any coordinator state,
    and return a DataflowDesc.
  * `import...into_dataflow` methods do not mutate any coordinator
    state, but modify the provided `DataflowDesc`.
  * neither the build nor import APIs take `catalog::{Item,View}`, as
    these catalog types are unnecessarily hard to construct for
    transient dataflows.
  * the `ship_dataflow` method is the only method that mutates
    coordinator state based on the provided `DataflowDesc`. It also
    broadcasts that `DataflowDesc` to the dataflow workers.

This isn't perfect, but it should make things much easier to reason
about, because the construction of the dataflow description is better
separated from updating coordinator state.
  • Loading branch information
benesch committed Aug 19, 2020
1 parent 97fe716 commit 2d7ea9b
Show file tree
Hide file tree
Showing 2 changed files with 90 additions and 143 deletions.
Loading

0 comments on commit 2d7ea9b

Please sign in to comment.