Some questions about reproduction.

[Peng-weil]

Thank you for your excellent work first!
When I reproduced your project, I found some discrepancies between the results and those presented in the paper, I ensured the file structure described in ReadMe and tried running "CUDA_VISIBLE_DEVICES=0 python train_net.py --config-file configs/ panoptic-segmentation/PADing.yaml --num-gpus 1 MODEL.WEIGHTS pretrained_weight_panoptic.pth", and the program executed successfully. If I understand correctly, the following should be the result in Table 1:

[09/25 03:31:55] PADing.evaluation.panoptic_evaluation_gzero INFO: Panoptic Evaluation Results:

	PQ	SQ	RQ	#categories
All	34.972	72.119	41.642	133
Things	42.356	77.451	49.989	80
Stuff	23.827	64.071	29.042	53
SEEN	37.847	74.862	45.048	119
UNSEEN	10.536	48.809	12.694	14

I'm not sure if there's some step I'm not doing right? I found at the beginning of the log
[09/24 15:46:05] fvcore.common.checkpoint WARNING: Some model parameters or buffers are not found in the checkpoint.
, and I'm not sure if it causes the results to be misaligned? Please feel free to let me know if there is anything I can provide. Thanks again for your work and for any help you can provide is appreciated.

[heshuting555]

Thank you for your interest!

What parameters are not found?
Besides, whether the dataset configuration is the same as the PADing?

[Peng-weil]

Thanks for the reply !!!

• I'm afraid I can't confirm the first question at the moment, I just saw this "WARNING" in the log, which I guess is a warning in the pre-training file. Could you please tell me if the log file in "https://drive.google.com/drive/folders/1QCD-0MXtGOJQdexjelOlOlx7MvUphaEZ" (found in another issue) is the one you posted? Because I found the same warning message in that log, and if "yes", I think this may not have serious consequences for the results?

• I think I have followed the PADing setup exactly. Because I successfully ran several of your " prepare dataset " code files and generated the corresponding paths and files.

Here are the paths I have set up, which I think should be fine.
** The number represents the amount of image files

├── coco
│   ├── annotations
│   │   ├── captions_train2017.json
│   │   ├── captions_val2017.json
│   │   ├── instances_train2017.json
│   │   ├── instances_val2017.json
│   │   ├── panoptic_train2017.json
│   │   ├── panoptic_val2017.json
│   │   ├── person_keypoints_train2017.json
│   │   ├── person_keypoints_val2017.json
│   │   ├── ZSIS
│   │   │   ├── instances_train2014_seen_48_17.json
│   │   │   ├── instances_train2014_seen_65_15.json
│   │   │   ├── instances_val2014_gzsi_48_17.json
│   │   │   ├── instances_val2014_gzsi_65_15.json
│   │   │   ├── instances_val2014_unseen_48_17.json
│   │   │   └── instances_val2014_unseen_65_15.json
│   │   └── ZSP
│   │   ├── panoptic_train2017_all.json
│   │   ├── panoptic_train2017_seen.json
│   │   ├── panoptic_train2017_unseen.json
│   │   ├── panoptic_val2017_all.json
│   │   ├── panoptic_val2017_seen.json
│   │   └── panoptic_val2017_unseen.json
│   ├── coco_stuff
│   │   ├── annotations
│   │   │   ├── train2017
│   │   │   │      ├──000000000009.png
│   │   │   │      └── ...118287
│   │   │   └── val2017
│   │   │            ├──000000000139.png
│   │   │            └── ...5000
│   │   ├── annotations_detectron2
│   │   │   ├── train2017
│   │   │   │      ├──000000000009.png
│   │   │   │      └── ...118287
│   │   │   ├── val2017_all
│   │   │   │      ├──000000000139.png
│   │   │   │      └── ...5000
│   │   │   ├── val2017_seen
│   │   │   │      ├──000000000139.png
│   │   │   │      └── ...5000
│   │   │   └── val2017_unseen
│   │   │            ├──000000000139.png
│   │   │            └── ...5000
│   │   ├── images (*same as ZegFormer)
│   │   │   ├── train2017
│   │   │   │      ├──000000000009.jpg
│   │   │   │      └── ...118287
│   │   │   └── val2017
│   │   │            ├──000000000139.jpg
│   │   │            └──...5000
│   │   ├── split
│   │   │   ├── all_classnames.json
│   │   │   ├── all_indexes.json
│   │   │   ├── novel_cls.npy
│   │   │   ├── seen_classnames.json
│   │   │   ├── seen_cls.npy
│   │   │   ├── seen_indexes.json
│   │   │   ├── unseen_classnames.json
│   │   │   ├── unseen_indexes.json
│   │   │   └── val_cls.npy
│   │   └── word_vectors
│   │   ├── fasttext.pkl
│   │   ├── glove.pkl
│   │   └── word2vec.pkl
│   ├── panoptic_semseg_train2017
│   │      ├──000000000009.png
│   │      └── ...118287
│   ├── panoptic_semseg_val2017
│   │      ├──000000000139.png
│   │      └── ...5000
│   ├── panoptic_train2017
│   │      ├──000000000009.png
│   │      └── ...118287
│   ├── panoptic_val2017
│   │      ├──000000000139.png
│   │      └── ...5000
│   ├── train2014
│   │      ├──COCO_train2014_000000000009.jpg
│   │      └── ...82783
│   ├── train2017
│   │      ├──000000000009.jpg
│   │      └── ...118287
│   ├── val2014
│   │      ├──COCO_val2014_000000000042.jpg
│   │      └── ...40504
│   └── val2017
│            ├──000000000139.jpg
│            └── ...5000
├── get_zsp_json.py
├── mscoco_unseen_classes.json
├── prepare_coco_semantic_annos_from_panoptic_annos.py
└── README.md

At the moment I'm assuming that the above logs were posted by you and reconfiguring the environment in Tesla-V100 (as described in the logs, which I was previously running in RTX) to reproduce it. I'm guessing maybe it's my (software or hardware) environment?

Thanks again for the help you have provided! I sincerely hope that my questions won't take up too much of your valuable time. : )

[Peng-weil]

log.txt

Here is my complete log if you would like to view it。

[heshuting555]

Maybe STRUCTURE has some variation for reproduction, you can try to set STRUCTURE_WEIGHT to a smaller value.

Thanks!

[Peng-weil]

Maybe STRUCTURE has some variation for reproduction, you can try to set STRUCTURE_WEIGHT to a smaller value.

Thanks!

Thanks for the guidance! I will try it soon!