[TASK][MEDIUM] Configurable NICs(Network Interface Cards) selection strategy

[pan3793]

What's the level of this task?

MEDIUM

Code of Conduct

• I agree to follow this project's Code of Conduct

Search before creating

• I have searched in the task list and found no similar tasks.

Mentor

• I have sufficient expertise on this task, and I volunteer to be a mentor of this task to guide contributors through the task.

Skill requirements

• Familiar with Kyuubi architecture
• Knowledge of Scala language and network programming

Background and Goals

In some environments, e.g. AliCloud ADB Spark cluster, it has multiple NICs(Network Interface Cards), Kyuubi uses findLocalInetAddress to find the first non-loopback address to bind, this may cause network issues. See details in #3409

Implementation steps

As title, we need flexible configurations to allow users to choose different NICs.

Note: The task level is set to MEDIUM because I want a flexible and extendable design, including docs and at least one real-world case.

Additional context

Introduction of 2024H1 Kyuubi Code Contribution Program