Runtime reorg #230
Runtime reorg #230
Conversation
Signed-off-by: David Wood <[email protected]>
Signed-off-by: David Wood <[email protected]>
Signed-off-by: David Wood <[email protected]>
Signed-off-by: David Wood <[email protected]>
Signed-off-by: David Wood <[email protected]>
|
@roytman approval or not I will wait on your ok before any merging of this. KFP v2 has priority so if this somehow conflicts (although it may also fix some issues), this can wait until kfp is merged. Of course, you should feel free to merge in my absence. |
@daw3rd thank you very much. |
| from data_processing_ray.runtime.ray import RayTransformLauncher | ||
| from data_processing_ray.runtime.ray.runtime_configuration import ( | ||
| RayTransformRuntimeConfiguration, | ||
| ) | ||
| from filter_transform import FilterTransformConfiguration |
There was a problem hiding this comment.
Oh, its a monster, that is based on a clever make files, but requires a lot of pycharm knowledge and very hard to understand and reason
Why are these changes needed?
Continued work on splitting transforms. For now this includes filter and some fixes for noop.
Extends the test-image target to not just run pytest, but to try and execute the transform using --help.
Fixes non-python transform runtimes by properly installing the adjacent python transform into the venv
Related issue number (if any).