[SUB-TASK][KPIP-2] Support YARN application-mode

[yanghua]

Code of Conduct

• I agree to follow this project's Code of Conduct

Search before asking

• I have searched in the issues and found no similar issues.

Describe the feature

This is a subtask of #1322

Motivation

No response

Describe the solution

No response

Additional context

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

[zuston]

Interested on it. Could u please assign this ticket to me? @yanghua

[pan3793]

This is a feature nice to have, not a blocker of 1.7

[link3280]

@yanghua @pan3793 I've created a POC link3280@a0c3dae and verified myself, PTAL. Codes related to the session mode are removed temporarily to simplify testing.

It needs some hacking to submit jobs from Kyuubi currently, but at the same time I'm preparing a FLIP to allow job graph submitted from outside the cluster in Flink application mode. If implemented, we should not need hacking anymore.

[pan3793]

Thank @link3280 for your excellent work.

[yaooqinn]

👍

[waywtdcc]

Can this get the job application id?

[link3280]

Can this get the job application id?

@waywtdcc The application id would be handled on the Flink side, thus we don't have to explicitly set an application id.