Auto Scene Discretization

[xiahongchi]

Hi!

Thanks for your great work! It's really amazing to see such great segmentation results. I wonder whether you have also implemented this automatically scene discretization in this code base?

[OceanYing]

Hi, Hongchi! Sorry for the late reply and thanks a lot for your interest!

We haven't implement the scene discretization in this code base. In order to do this, you could try to apply clustering methods (like HDBScan) on the resulting semantic vectors of each Gaussian point.

[weijielyu]

Hi, thanks for the hint! I wonder if I use HDBScan to cluster each Gaussian point, how to render segmentation masks with cluster ID? Meanwhile what's the parameters setting for HDBScan? Thanks!

[OceanYing]

1. One idea is, after clustering Gaussian point to different groups, you can assign each Gaussian point $p_i$ with an updated semantic feature (the average feature $\bar{f_j}$ of the group $j$ that $p_i$ belongs to). Then you can render new semantic images from the updated Gaussian points. Next, for each image, the segmentation mask of object with cluster ID $j$ can be obtained by calculating a similarity map (feature distance between each rendered pixel feature and the average feature $\bar{f_j}$) and thresholding it.
2. The parameters of HDBScan need to be tuned on to your demand. (e.g. granularity.)