💫 NCP 💫

Pytorch code for "NCP: Neural Correspondence Prior for Effective Unsupervised Shape Matching" - NeurIPS 2022

TLDR: Using noisy point-to-point (P2P) maps to train a non-rigid shape-matching pipeline in a supervised fashion results in P2P mappings of superior quality compared to the training data.

---

👷 Installation

This implementation requires Python >= 3.7. Install dependencies using pip:

pip install -r requirements.txt

📖 Usage

In this repository, we provide Jupyter notebooks that reproduce the main insights of our paper. Additionally, we include the code for our FSKD algorithm and various losses used in our experiments.

Our paper offers new insights into shape matching rather than presenting a new architecture. To experiment with these insights, we utilized the Deep Functional Maps framework. We recommend users to apply their favorite implementations of Deep Functional Maps. Recommended implementations include Diffusion-Net, DPFM, and SRFeat. In our experiments, we used the Diffusion-Net implementation.

To leverage the NCP effect, obtain some noisy point-to-point maps (for example, using unsupervised methods) and use them to train your network with the method we proposed in the paper. The output maps generated will be of higher quality than those used for training.

• In noise_impedance.ipynb, we provide an illustration of the NCP effect, and provide the code to reproduce the results of Figure 2 in the paper.
• In test_time_denoising.ipynb, we provide the code for the test time denoising algorithm explained in Section 4.2 of the paper.
• In fskd.py, we provide the code for the FSKD algorithm.
• In losses.py, we provide the code for the losses used in our experiments.

📈 Results

If you wish to report our result, we have summarized them below. Our method is referred to as NCP in the case of shape matching, and FSKD for keypoint detection. X on Y indicates that the method was trained on dataset X and tested on dataset Y.

• Near Isometric Shape Matching: We provide results on the FAUST (F), Scape (S) datasets. We used the remeshed version of the datasets. We report the mean geodesic error, following the protocol used in all deep functional map papers. Our method is unsupervised.

Method	F on F	S on S	F on S	S on F
NCP	3.0	3.5	4.2	2.9
NCP + ZoomOut	1.9	2.4	2.6	1.9

• Non-Isometric Shape Matching: We provide results on the SMAL and SHREC’20 datasets. We report the mean geodesic error, following the same protocol as in all the deep functional maps papers. Our method is unsupervised.

Method	SMAL	SHREC’20
NCP	5.8	8.5

• Point Cloud Matching: We provide results on the KeyPointNet dataset, using the same evaluation protocol as in the original paper. Our method is unsupervised.

Method	Airplane	Bathtub	Bed	Bottle	Cap	Car	Chair	Guitar	Helmet	Knife	Laptop	Motor	Mug	Skate	Table	Vessel
NCP	2.3	4.3	3.7	5.6	3.2	3.7	4.4	3.3	6.7	7.5	1.9	3.6	3.8	3.4	3.2	10.5

• Part Segmentation Transfer: We provide results on the PartNet dataset. We report the mean IoU, following the same protocol as in the original paper. Our method is unsupervised.

Method	pla.	bag	cap	car	cha.	ear.	gui.	kni.	lam.	lap.	bik.	mug.	pis.	roc.	ska.	tab.	avg.
NCP	63.7	66.7	68.7	57.4	80.2	59	78.8	72.5	61.9	91.4	57.2	89.5	61.4	44.2	63.6	79.2	69.2

• Keypoint Detection: We provide results on the KeyPointNet dataset. We report the mean IoU. We compare our method against learning based algorithms. Our method uses 3 labeled source shapes (three-shot keypoint detection).

🎓 Citation

If you find this work useful in your research, please consider citing:

@inproceedings{attaiki2022ncp,
    title={{NCP}: Neural Correspondence Prior for Effective Unsupervised Shape Matching},
    author={Souhaib Attaiki and Maks Ovsjanikov},
    booktitle={Advances in Neural Information Processing Systems},
    year={2022}
}