Proposals for processes / Move some processes from stable to proposals

[m-mohr]

Background

We have specified a whole range of processes in openEO, mostly during a meeting two years ago in Belgium. We had no real clue yet what we'd need and we also didn't had established a procedure for "proposals" (i.e. everything added for the processes was directly part of the specification).

Using the Hub, I made an overview of the processes not yet implemented at back-ends:

• aggregate_spatial_binary (marked as experimental anyway)
• cummax
• cummin
• cumproduct
• cumsum
• debug
• filter_labels
• load_result
• load_uploaded_files
• reduce_dimension_binary (marked as experimental anyway)
• resample_cube_temporal (implemented once by @clausmichele, but he reported issues resample_cube_temporal behavior openeo-processes#194 and in a recent openEO dev telco we discussed that the process needs breaking changes to be useful)
• run_udf_externally (marked as experimental anyway)

Proposal

I'm proposing the following things:

1. Add a proposals folder (already done in draft though) that includes all new and (currently) experimental processes.
   • All new processes are added to this folder.
   • Processes will only be moved from proposals to the normative specification once there are at least two implementations and an example process graph in the examples folder showing it in a use case. This doesn't require a PSC vote individually as it's not a breaking change, just an addition. Nevertheless, each release for the processes will go through PSC vote.
   • The proposals folder allows "breaking changes" without a PSC vote and without increasing the major version number (e.g. a breaking change here doesn't require us to make the next version number 2.0.0...)
   • The proposals are released as experimental processes with the other processes.
   • (Breaking changes in the "stable" processes [i.e. in the root folder of the repo] still need a PSC vote - this is not a change, just want to mention it so that it's clear to everyone.)
2. Move all processes mentioned above to the "proposals" folder. Strictly speaking this could be a breaking change. Not moving them itself, but changes in the proposals section later on would be somewhat unexpected. I think this can be mitigated as the processes are (mostly) not yet implemented or experimental (see the list above).

This behavior would make sure that the implementations can experiment more and "stable" processes are driven by implementations and use cases. Also, it likely reduces the number of votes we need here for things that may often be a bit too technical/detailed/... for discussion in this group.

This is how I'd describe the new structure in the Repository section of the README:

• The *.json files provide the stable process specifications as defined by openEO. Stable processes need at least two implementations and a use-case example added to the examples folder or consensus from the openEO PSC.
• The *.json files in the proposals folder provide proposed new process specifications that are still experimental and subject to change, including breaking changes. Everyone is encouraged to base their work on the proposals and give feedback so that eventually the processes evolve into stable process specifications.

Additional context

This is the original issue: Open-EO/openeo-processes#207

Note: As you may have realized, for existing "stable" processes one implementation is enough for now, otherwise we'd need to move additional processes. But at least all other processes went through intensive discussions in the openEO project phase.

Deadline: 19.01.2020 (I don't expect any votes in the next two weeks ;-) )

[m-mohr]

+1

[mkadunc]

+1

[edzer]

Great plan. +1

[clausmichele]

+1 for me too. Maybe in the future we can also clarify this part:
(Breaking changes in the "stable" processes [i.e. in the root folder of the repo] still need a PSC vote - this is not a change, just want to mention it so that it's clear to everyone.)
which I thinks it's the case if we want to modify the behavior of resample_cube_temporal.

[m-mohr]

@clausmichele How would you like to clarify it?
With the proposal above, resample_cube_temporal would be moved to proposals and there we can change it until it works in two implementations.

[aljacob]

+1

[lforesta]

+1

[jdries]

+1

[neteler]

+1

[m-mohr]

This got accepted. Thank you all and happy holidays.