[DPOTrainer] Fix peft + DPO + bf16 if one uses generate_during_eval or pre-computed logits

[younesbelkada]

What does this PR do ?

Fixes #1202 which is a regression

#1143 introduced peft_module_casting_to_bf16 for DPOTrainer which worked fine when generate_during_eval is set to False.

peft_module_casting_to_bf16 will cast the LayerNorm layers in fp32 for stability purposes for smoother training, in case one uses peft + 4-bit quantization (QLoRA).

If one uses bf16=True in the TrainingArguments, Trainer will automatcially compute the loss under the torch.cuda.amp.autocast regime, making the forward pass possible to work without any dtype mismatch issue

However, we need to make sure all other methods inside DPOTrainer that call the model's forward pass will also use that context manager to make sure there is no dtype mismatch

since this is a regression, will publish a patch release after merging this pr

cc @kashif @pacman100 @lvwerra

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[kashif]

thanks @younesbelkada awesome!