fix some parameter setting does not work for CPU DDP and bf16 fail in…#527

Merged

sgugger merged 3 commits intohuggingface:mainfrom sywangyi:ddp_cpu_fix

Jul 18, 2022

+10-2

Commits on Jul 18, 2022

fix some parameter setting does not work for CPU DDP and bf16 fail in DDP path
sywangyi
committed
if number_machine > 1, get the ip and port accelerate config set
sywangyi
committed
if main_process_ip and port is set by user, use them, else use default "127.0.0.1" when DDP is used in one machine
sywangyi
committed