Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
coord: adjust dataflow building + shipping APIs
Reconfiguring the APIs for building and shipping DataflowDescs results in much simpler call sites, and hopefully easier-to-understand mutation patterns. In the new API: * `build...dataflow` methods do not mutate any coordinator state, and return a DataflowDesc. * `import...into_dataflow` methods do not mutate any coordinator state, but modify the provided `DataflowDesc`. * neither the build nor import APIs take `catalog::{Item,View}`, as these catalog types are unnecessarily hard to construct for transient dataflows. * the `ship_dataflow` method is the only method that mutates coordinator state based on the provided `DataflowDesc`. It also broadcasts that `DataflowDesc` to the dataflow workers. This isn't perfect, but it should make things much easier to reason about, because the construction of the dataflow description is better separated from updating coordinator state.
- Loading branch information