Skip to content

Commit

Permalink
[SW-198498] pass "lazy_mode" arg to GaudiLlamaModel GaudiTrainer
Browse files Browse the repository at this point in the history
Problem: TrainingArgs.use_lazy_mode is not used by GaudiLlamaModel

Cause: lazy_mode argument was not passed by GaudiTrainer

Solution: Added missing argument to inputs in
   GaudiTrainer._inner_training_loop

Change-Id: I956023956af3d7962b24be53ec74d20e6bb56bd6
  • Loading branch information
mlapinskix authored and astachowiczhabana committed Sep 27, 2024
1 parent b6a2f68 commit 6f41803
Showing 1 changed file with 3 additions and 1 deletion.
4 changes: 3 additions & 1 deletion optimum/habana/transformers/trainer.py
Original file line number Diff line number Diff line change
Expand Up @@ -982,7 +982,9 @@ def hpu_deepspeed_checkpointing(function, *checkpoint_args, use_reentrant: Optio
inputs["flash_attention_recompute"] = True
if self.model.generation_config.flash_attention_causal_mask:
inputs["flash_attention_causal_mask"] = True

if self.model.config is not None:
if self.model.config.model_type in ["llama", "qwen2", "mistral", "starcoder2"]:
inputs["lazy_mode"] = args.use_lazy_mode
# TODO: keep syncs for fast DDP?
with self.accelerator.accumulate(model):
tr_loss_step = self.training_step(model, inputs)
Expand Down

0 comments on commit 6f41803

Please sign in to comment.