Learning Motion and Temporal Cues for Unsupervised Video Object Segmentation(Under Review)

Note

Thank you for your interest in our work. Currently, our paper is under review, and this repository contains only the test code. We are actively working to prepare the complete codebase, which will include both training and testing phases. We will release the full code soon.

Demo

Get Started

Environment

• python == 3.8.15
• torch == 1.10.0
• torchvision == 0.11.0
• cuda == 11.4
• opencv == 4.6.0

Datasets

Please download the following datasets:

UVOS datasets:

• YouTube-VOS: YouTube-VOS
• DAVIS: DAVIS
• YouTube-Objects: YouTube-Objects
• FBMS: FBMS
• LongVideos: LongVideos

VSOD datasets:

• DAVIS: same as UVOS.
• DAVSOD: DAVSOD
• SegTrack-V2: SegTrack-V2
• ViSal: ViSal

To quickly reproduce our results, we upload the processed data to Google Drive and Baidu Disk(code: qcbh).

Models

stage	model link
pre-train	Google Drive, Baidu Disk(code: qcbh)
fine-tuning	Google Drive, Baidu Disk(code: qcbh)

To reproduct the results we reported in paper, please download the corresponding models and run test script.

Training

Waiting

Testing

Download the trained MTNet, and placing it in the ./saves or anywhere.

python test.py [test_model] [task_name] [test_dataset] [output_dir]

Testing for UVOS task:

python test.py --test_model ./saves/mtnet.pth --task_name UVOS --test_dataset DAVIS16 --output_dir output

Testing for VSOD task:

python test.py --test_model ./saves/mtnet.pth --task_name VSOD --test_dataset DAVIS16 --output_dir output

Results

Precomputed outputs - Google Drive

Precomputed outputs - Baidu Disk(code: qcbh)

Evaluation

Evaluation for UVOS results:

python test_scripts/test_for_davis.py --gt_path ../data/DAVIS16/val/mask --result_path output/MTNet/UVOS/DAVIS16/

Evaluation for VSOD results:

python test_scripts/test_vsod/main.py --method MTNet --dataset DAVIS16 --gt_dir test_scripts/test_vsod/gt/ --pred_dir test_scripts/test_vsod/results/

Visualization

Specify the dataset in visualize.py, then run:

python visualize.py

References

This repository owes its existence to the exceptional contributions of other projects:

• STCN: https://github.com/hkchengrex/STCN
• AOT: https://github.com/yoxu515/aot-benchmark
• HFAN: https://github.com/NUST-Machine-Intelligence-Laboratory/HFAN
• FSNet: https://github.com/GewelsJI/FSNet
• AMCNet: https://github.com/isyangshu/AMC-Net
• DAVSOD: https://github.com/DengPingFan/DAVSOD

Many thanks to their invaluable contributions.