[FEATURE] A Universal ML command in PPL for train/predict/trainandpredict

[jngz-es]

Is your feature request related to a problem?
Currently for each new algorithm in ml-commons, we have to add a new command in PPL which means we have to implement an entire PPL command process including syntax parser, logical plan and physical plan. It is very inefficient for development.
From user interface perspective, it is more clean and reasonable to have one command for all algorithms than each command for each algorithm.

What solution would you like?
We want to provide only one PPL command (ml) for all algorithms in ml-commons about train/predict/trainandpredict. So for new algorithm launch, we just need to add some changes in ml-commons part, don't need to touch PPL plugin part any more.

What alternatives have you considered?
We considered add 3 commands each for train, predict and trainandpredict in terms of ml-commons APIs.

Do you have any additional context?
We plan to keep the existing algorithms PPL commands at this moment, but want to deprecate them in the future.

Example
ml action=train algo=kmeans centroids=3 iterations=2 distance_type='cosine'

[dai-chen]

@jngz-es I've marked this as RFC. If needed, you can put some query examples for community feedback. Thanks!

[jngz-es]

@jngz-es I've marked this as RFC. If needed, you can put some query examples for community feedback. Thanks!

Added an example query, thanks!

[ahopp]

Looks like I'm very late on this, but I think its important to provide justification on why this feature is being developed/have been developed in PPL but not in parity SQL. I assume you all chose PPL because it was easier and/or because you all needed it downstream (e.g., PPL in observability) but I think it's important to share this justification with the community given the adoption (i.e., SQL versus PPL). I realize it is used heavily in the observability plugin experience and if that's the justification, we should be clear.

[dai-chen]

I assume we can close this. If anything else, please open new issue labeled with future release version. Thanks!