AMD: Autoregressive Motion Diffusion

AMD: Autoregressive Motion Diffusion

Bo Han  Hao Peng  Minjing Dong  Yi Ren  Yixuan Shen  Chang Xu 
Zhejiang University   Unity China   University of Sydney   National University of Singapore

📜 TODO List

• Release the HumanLong3D Dataset
• Release the HumanMusic Dataset
• Release the main codes for implementation.

⚙️ Content

📁 Code/
├── data_loaders/                      # Data loading related
│   ├── humanml/
│   │   ├── data/
│   │   │   └── dataset.py
│   │   └── utils/
│   │       └── plot_script.py        # Skeleton visualization
│   ├── get_data.py
│   └── tensors.py
│
├── diffusion/                         # Core diffusion model
│   └── gaussian_diffusion.py
│
├── eval/                              # Evaluation scripts
│   ├── eval.py                       # Single motion evaluation
│   ├── eval_AUTOREG.py              # Compound motion evaluation
│   └── eval_T2L.py                  # Motion duration prediction network evaluation
│
├── model/                             # Model definitions
│   ├── amd_autoreg.py               # Compound motion model
│   └── amd.py                       # Single motion model
│
├── train/                             # Training scripts
│   ├── train_amd_single.py          # Single motion training
│   └── train_amd_autoreg.py         # Compound motion training
│
├── utils/                             # Utility functions
│   ├── parser_util.py               # Parse running parameters
│   └── model_util.py                # Parse model parameters
│
├── visualize/                         # Visualization tools
│   └── joints2smpl                   # Skeleton to SMPL conversion
│
├── text2length.py                     # Motion duration prediction
├── 0_amd_single_generate.py          # Single motion generation
└── 1_amd_autoreg_generate.py         # Compound motion generation

🏃 Training

Single Motion Model

nohup python -m train.train_amd_single \
    --save_dir save/0_humanml3d_single \
    --data_dir ./dataset/HumanLong3D \
    --device 1 \
    --overwrite \
    > ./save/0_humanlong3d_single/train.log 2>&1 &

Compound Motion Model

nohup python -m train.train_amd_autoreg \
    --save_dir save/0_humanlong3d_autoreg \
    --data_dir ./dataset/HumanLong3D \
    --device 0 \
    --overwrite \
    > ./save/0_humanlong_autoreg/train.log 2>&1 &

Motion Duration Predictor

nohup python train_length_est.py \
    --name t2l \
    --gpu_id 2 \
    --dataset_name t2m \
    > ./checkpoints/t2m/train.log 2>&1 &

📊 Evaluation

Single Motion Evaluation

Without Duration Prediction

nohup python -m eval.eval \
    --model_path '' \
    --eval_mode mm_short \
    --device 0 \
    > ./save/xxx/0_eval_mm.log 2>&1 &

With Duration Prediction

nohup python -m eval.eval_T2L \
    --model_path '' \
    --eval_mode mm_short \
    --device 1 \
    > ./save/xxx/0_eval_mm_T2L.log 2>&1 &

Compound Motion Evaluation

nohup python -m eval.eval_AUTOREG \
    --model_path '' \
    --eval_mode mm_short \
    --device 3 \
    > ./save/xxx/0_eval_mm_AUTOREG.log 2>&1 &

Duration Predictor Evaluation

python eval_length_est.py \
    --name t2l \
    --gpu_id 0 \
    --dataset_name t2m

🎮 Synthesis

Single Motion Generation

python 0_amd_single_generate.py \
    --model_path 'path/to/your/model' \
    --text "a person is walking" \
    --device 0

Compound Motion Generation

python 1_amd_autoreg_generate.py \
    --model_path 'path/to/your/model' \
    --text "a person walks forward then jumps" \
    --device 0

🤝 Citation

If you find this repository useful for your work, please consider citing it as follows:

@article{Han2024,
  title={AMD: Autoregressive Motion Diffusion},
  author={Bo Han, Hao Peng, Minjing Dong, Yi Ren, Yixuan Shen, Chang Xu},
  journal={AAAI},
  year={2024}
}