[BUG] Gather Stat is not working for DeepSpeed?

[allanj]

I guess the issue is mainly caused by this line:

trl/trl/trainer/ppo_trainer.py

Line 333 in 6ff0fac

self.is_distributed = self.accelerator.distributed_type == "MULTI_GPU"

The self.accelerator.distributed_type is DEEPSPEED when I use DeepSpeed Zero stage 2 with accelerate.

It works when I change this line to

self.is_distributed = self.accelerator.distributed_type == "MULTI_GPU"  or self.accelerator.distributed_type == "DEEPSPEED"

But I think, to make this more general, we can try something like

self.is_distributed = self.accelerator.num_process > 1

Any suggestions?

[seanexp]

looks like a valid fix to me! 👍

[allanj]

maybe I will try to make a pull request for this. Also, I feel like there are some redundant for loop that we can avoid in the code

[younesbelkada]

Hi everyone, yes this looks like a reasonable change, would you be happy to open a quick PR for it?

[allanj]

Hi everyone, yes this looks like a reasonable change, would you be happy to open a quick PR for it?

I have created the simple PR fix. Please help review: #997

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.