feat(sdk): Add Input Parameter support for configmap, secrets, node selectors, tolerations, pull secrets

[HumairAK]

Description of your changes:

Related:

• [kubernetes_platform] Error calling kubernetes.use_secret_as_env and use_secret_as_volume with secret name as parameter #10914
• [feature] Add ability to specify node affinity & toleration using KFP V2 #9682
• [sdk] Unable to pass env variables as pipeline parameters #11395
• [feature] Make set_env_variable compatible with the channels [KFP V2] #10111
• [backend] Cannot inject environment variables and volumes into components with the KFP CLI #9924
• [feature] Support parameter inputs for V2 Kubernetes_platform Spec #10534

These will be marked resolved once the backend & sdk changes are merged after this.

This change provides proto api support for the following kubernetes platform
fields:

• configmap name for configmap as env & volume
• secret name for configmap as env & volume
• image pull secret name
• toleration json
• node selector json

Checklist:

• You have signed off your commits
• The title for your pull request (PR) should follow our title convention. Learn more about the pull request title convention used in this repository.

[HumairAK]

cc @chensun please review the proto changes here when you get the chance so we are aligned here, as some fields are also being deprecated

[HumairAK]

Please also note that this is a more generic implementation of how we were handling inputs already for pvc mounts in the k8s platform spec here, but extended to various other fields

One thing also worth noting is that we probably don't need the entire InputParameterSpec as it is defined in the pipelinespec, for example TaskFinalStatus might not make much sense here, but for simplicity and maintainability's sake I figured it's easier to just keep them fully in sync. In the future we should consider just updating the structure of our proto files so we can directly import these instead of copying them over.

[HumairAK]

CI failures are due to flaky tests I believe. Will retry them a couple of times to confirm.
Actually I don't think so, investigating.

[HumairAK]

The tests failures are a redherring, it's because of the compiled proto packages that are not utilizing the new code
See this test PR that references the k8s go packages locally, the tests here pass, the only difference being this commit

[HumairAK]

cc @chensun wdyt? any objections to merging this? Once done I'll follow up with backend PR.

[chensun]

/lgtm

[mprahl]

Is this still needed?

[HumairAK]

well we need this line to be recognizable in kubernetes_executor_config_pb2.py (the python proto output):

import pipeline_spec_pb2 as pipeline__spec__pb2

here python will not be able to find this module because we're outputing it in the same location

another maybe a more preferable approach can be to just create the module directory with an __init__.py and have the output go there, this should get it to work and we can drop these lines

I don't have a strong preference

[google-oss-prow]

New changes are detected. LGTM label has been removed.

[google-oss-prow]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please ask for approval from chensun. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• .github/OWNERS
• api/OWNERS
• kubernetes_platform/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment