Distributed Inference test

[mikekgfb]

run distributed inference test

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/torchchat/1501

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 2 New Failures, 1 Unrelated Failure

As of commit 0b992a4 with merge base 2766a95 ():

NEW FAILURES - The following jobs have failed:

• pull / test-gpu-eval-sanity-check (cuda, stories15M) / linux-job (gh)
  RuntimeError: Command docker exec -t df67ce14bf95b3393b7a342b665537eb60695b1a45b0ff88d304ed7cf15c46db /exec failed with exit code 1
• Run the README instructions - with stories / test-distributed-cuda / linux-job (gh)
  RuntimeError: Command docker exec -t 7da94d3d671be19c198c74cb4fd94c60820ad0886cc7b995bb07686bb0a40273 /exec failed with exit code 1

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• pull / test-torchao-aoti-experimental (macos-14-xlarge) (gh) (trunk failure)
  Process completed with exit code 134.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[mikekgfb]

@lessw2020 @kwen2501 please have a look at this distributed test failure?

https://github.com/pytorch/torchchat/actions/runs/13502875993/job/37737567075

++ python3 torchchat.py generate stories110M --distributed --tp 2 --pp 2 --prompt 'write me a story about a boy and his bear'
[...]
  [rank0]: torch.distributed.DistBackendError: NCCL error in: /pytorch/torch/csrc/distributed/c10d/NCCLUtils.cpp:77, invalid usage (run with NCCL_DEBUG=WARN for details), NCCL version 2.21.5
  [rank0]: ncclInvalidUsage: This usually reflects invalid usage of NCCL library.
  [rank0]: Last error:
  [rank0]: Duplicate GPU detected : rank 0 and rank 1 both on CUDA device 1e0