[V1][Help Wanted] Porting missing sampling parameters to V1

[WoosukKwon]

Anything you want to discuss about vllm.

To switch the engine from V0 to V1, we need to comprehensively support the sampling parameters in https://github.com/vllm-project/vllm/blob/main/vllm/sampling_params.py

While most of the key parameters are already supported, some of them are missing:

TODO (help wanted):

• n (parallel sampling) [V1] V1 engine implements parallel sampling (AsyncLLM and LLMEngine) #10980 @afeldman-nm
• guided_decoding (structured decoding) [V1][Core] Structured decoding #12388 @aarnphm
• logit_bias Support logit_bias in v1 Sampler #13079 @houseroad
• min_p [V1][Core] min_p sampling support #13191 @AoyuQC
• bad_words (originally implemented via logits processor) [V1] Support bad_words in sampler #13376 @22quinn
• allowed_token_ids (originally implemented via logits processor) [v1] Support allowed_token_ids in v1 Sampler #13210 @houseroad

Parameters that will not be supported in V1:

• best_of
• logits_processors

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[22quinn]

Seems like good first issues - @WoosukKwon I'd like to take a stab at bad_words

[akeshet]

Parameters that will not be supported in V1:

...
logits_processors

Is this to say that support for logits_processors is going to be permanently dropped in V1, or is it just out of scope for this particular GH issue?

[WoosukKwon]

@22quinn Thanks for volunteering! Could you please submit a PR by EoW?

[WoosukKwon]

@akeshet

We plan to re-design the API for that. We will probably not allow per-request logits processor (because this is too complex and slow). We are exploring other options.

Please refer to this comment from @njhill: #12688 (comment)

[njhill]

We'll still have a logits processor plugin mechanism (not per request) but the interface will be different. I think that @AlpinDale was working on a proposal for this.

It might be nice for some of these sampling parameters to be implemented via that same abstraction but assuming it won't be ready before we need them we can plan to retrofit where it makes sense.

[maliknaik16]

@22quinn, are you still working on this task? If not, I'd be happy to take it up! Let me know what you think.

cc @WoosukKwon

[WoosukKwon]

@maliknaik16 Please feel free to take it!

@22quinn Let us know if you already have the PR.

[22quinn]

@maliknaik16 @WoosukKwon I'm on it, PR will be up soon

[WoosukKwon]

@22quinn Oh great. Thanks!