[llm.serving] Address dependencies related issues for llm serving

[GeneDer]

Why are these changes needed?

Two dependencies we are resolving that requires to pin xgrammar and pynvml to specific versions. Related vllm PR/ issues

• [Bugfix] Pin xgrammar to 0.1.11 vllm-project/vllm#13338
• [Bug]: RuntimeError: Failed to infer device type with v0.7.2 vllm-project/vllm#12847

Related issue number

Checks

• I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
• I've run scripts/format.sh to lint the changes in this PR.
• I've included any doc changes needed for https://docs.ray.io/en/master/.
  • I've added any new APIs to the API Reference. For example, if I added a
    method in Tune, I've added it in doc/source/tune/api/ under the
    corresponding .rst file.
• I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
• Testing Strategy
  • Unit tests
  • Release tests
  • This PR is not tested :(

[GeneDer]

cc @kouroshHakha

[aslonnie]

(no action required)

this is fine, you can also just pin the version in the test here.

the test requirement is not exposed to user, so it is easy to update.

the non test requirement is saved in the wheel, and will become immutable after package release.

you might want to add things like !=0.1.13 in ray[llm]'s requirement, so that pip install ray[llm] has a higher chance to work for user.

[aslonnie]

llm tests are passing and llm images are building. I am merging this for you.