Weigths of Vit Base 16

[FXLYZ]

Hi, it is a wonderful work in surgical domain!
I want to use the weights of vit base 16, and I don’t know how to change the config to download it. Because the default image backbone name is Resnet_50.

Could you help me solve this problem?

[Flaick]

Hello, thanks for your appreciation!
We currently do not provide the ViT based model for the following reasons:

1. ViT's training on a relatively "small" dataset, i.e., SVL-Pretrain, is not stable and highly relies on initialization. For example, the model initialized from Dino works better than ImageNet.
2. The model easily overfits to some patterns and does not outperforms the RN50 for some cases.
   But we have seen a potential benefit of Vision Transformer backbone and we will release one soon, which should be comparable to our newest model (PeskaVLP)
   Thanks,
   Kun