Training and evaluation environment: Python 3.9, PyTorch 1.13.1, CUDA 11.0. Run the following command to install required packages.
pip3 install -r requirements.txt
You need to configue the paths to the datasets in config.yml before training or testing.
A script download_datasets.sh is prepared to download and organize required datasets.
| Dataset | Description | Download Link |
|---|---|---|
| MS COCO | 118k images with 1.2M instances (train) | official site |
| LVIS v1.0 | 100k images with 1.2M instances (total) | official site |
| COCO+LVIS* | 99k images with 1.5M instances (train) | original LVIS images + combined annotations |
| SBD | 8498 images with 20172 instances for (train) 2857 images with 6671 instances for (test) |
official site |
| Grab Cut | 50 images with one object each (test) | GrabCut.zip (11 MB) |
| Berkeley | 96 images with 100 instances (test) | Berkeley.zip (7 MB) |
| DAVIS | 345 images with one object each (test) | DAVIS.zip (43 MB) |
| Pascal VOC | 1449 images with 3417 instances (test) | official site |
| COCO_MVal | 800 images with 800 instances (test) | COCO_MVal.zip (127 MB) |
An example script to run the demo.
python demo.py --checkpoint=weights/cocolvis_icl_vit_huge.pth --gpu 0
Before evaluation, please download the datasets and models, and then configure the path in config.yml.
Download our model, please download below 3 zipped files and extract before use:
Use the following code to evaluate the huge model.
python scripts/evaluate_model.py NoBRS \
--gpu=0 \
--checkpoint=cocolvis_icl_vit_huge.pth \
--datasets=GrabCut,Berkeley,DAVIS,PascalVOC,SBD \\
--cf-n=4 \\
--acf
# cf-n: CFR steps
# acf: adaptive CFR
Before training, please download the MAE pretrained weights (click to download: ViT-Base, ViT-Large, ViT-Huge) and configure the dowloaded path in config.yml
Please also download the pretrained SimpleClick models from here.
Use the following code to train a huge model on C+L:
python train.py models/plainvit_huge448_cocolvis.py \
--batch-size=32 \
--ngpus=4
@article{sun2023cfricl,
title={CFR-ICL: Cascade-Forward Refinement with Iterative Click Loss for Interactive Image Segmentation},
author={Shoukun Sun and Min Xian and Fei Xu and Tiankai Yao and Luca Capriotti},
year={2023},
eprint={2303.05620},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
Our project is developed based on RITM and SimpleClick