[LoRA] Adds support for bias in LoRA

[followumesh]

Motivation
PEFT, https://github.com/foundation-model-stack/fms-hf-tuning includes support for tuning LoRA bias. This PR enables bias for lora, so the adapters with bias will work with vLLM.

Changes Included

• LoRA bias support for different types of modules.
• LoRA bias support for fully sharded LoRA.
• Test file test-lora-bias.py

[Yard1]

Could we add an argument to the engine enable_lora_bias and avoid initializing the bias tensors if it's false? If the user knows none of their loras will have bias, we can save memory.

[followumesh]

@Yard1 Thanks for reviewing the PR. I have added the enable_lora_bias flag (default set to false), which prevents the allocation of lora bias tensors when false.

[njhill]

Related: #5930

[Yard1]

Looks good, can we also add an e2e test?

[DarkLight1337]

To speed up the CI queue for #5905, I've cancelled the distributed tests for the latest CI run in this PR since they won't pass anyway until #5905 has been merged. Please merge main into your branch after that happens so that the CI can pass once again.

[followumesh]

@Yard1 Thanks for reviewing. I've added an e2e test for the lora_bias support.

[njhill]

@followumesh you need to run ./format.sh to fix the linting errors

[followumesh]

@njhill I have addressed your comments above. Can you please review this again? Thanks

[njhill]

Thanks @followumesh and sorry for the delay.

There's one remaining small but major thing to fix (and tests are failing due to this).

[njhill]

This doesn't look right and is causing blanket lora failures. I think it should be:

Suggested change

	if not self.lora_config.bias_enabled:
	module_lora.bias = None
	raise ValueError(
	f"Adapter bias cannot be used for {module_name}"
	" without --enable-lora-bias.")
	if module_lora.bias is not None and not self.lora_config.bias_enabled:
	raise ValueError(
	f"Adapter bias cannot be used for {module_name}"
	" without --enable-lora-bias.")

[followumesh]

Incorporated the comment.

[njhill]

Hmm OK fair enough, I guess the typing errors are preexisting.

[njhill]

Suggesting small change here to cover case that bias is a tensor rather than a list (from a typing pov the lora_module could be a LoRALayerWeights rather than a PackedLoRALayerWeights ... not sure whether that will ever be the case in practice but no harm in having the check here cover it).

Also suggest a small change to the comment.

[njhill]

Thanks @followumesh!

[njhill]

@followumesh there are a few failures in the existing LoRA tests which look related.

[followumesh]

@njhill All LoRA tests are sucessful now.

[jeejeelee]

Thanks for complete this feature. I have two question about this featue:

• Is this feature compatible with PEFT?
• Have you done any benchmarking? Adding --enable-lora-bias seems to inevitably impact performance.