Feature Request: Make the workflow files more compact

[EshaanAgg]

Feature Description

Currently, nearly 100 workflow files are present in the repository as of now, some of which are almost similar except for a couple of parameters. Some examples of the same include:

We can make use of the matrix feature provided by GitHub workflows just to set the parameters of a job, and then use them. This would allow us to run multiple jobs but have a common definition for them.

Use Case(s)

IMO, this approach is more maintainable and easier to debug, as now, if we find a bug or update any workflow, we need not copy the same to all the similar ones. Also, it makes it easier to add and remove versions on which we want to run the workflows. Selective adoption of this approach can help to reduce the duplication in the codebase.

[EshaanAgg]

I would like to create a proof of concept for the cluster_end_to_end_XX workflows if the maintainers find the same to be relevant!

[frouioui]

I think it could be relevant to combine certain workflow files into one. However, the matrix needs to be configured in a way that the whole workflow does not fail if a given job fails. Moreover, most of the workflow files are auto-generated with the make generate_ci_workflows command, we would need to modify ./test/ci_workflow_gen.go in a way that it is still easy for us to configure what workflows we want to run.