Please add another interpolation mode for NiftiSaver to avoid ziazag pattern caused by 'nearest'

[Jingnan-Jia]

Is your feature request related to a problem? Please describe.
In the current NiftiSaver, there are only two interpolation modes: "nereast" and "bilinear". It is not enough. Detailed explanication as follows.

If I have an lung CT image with shape (1024, 512, 512). To do lung or lobe segmentation, I need to down sample the original CT image to a smaller size (256, 256, 256). So the resample factors are (0.25, 0.5, 0.5). After I finish the model training, the predicted lung or lobe segmentation needs to be up-sampled to the original shape during inference. If I use "bilinear" mode for NiftiSaver, some zigzag pattern would appear along the edge of lung or lobe.

Describe the solution you'd like
Using current MONAI, I came up with several possible solutions to avoid the zigzag pattern in NiftiSaver :

1. Use "bilinear" mode in NiftiSaver at first, and then use a threshold of 0.5 to binarize the results. However this method only applies to binary segmentation tasks, like lung segmentation. In multi-label segmentation, there is not a proper threshold for the whole image.
2. I can use "nearest" mode at first to get the output with zigzag pattern. Then use some smooth algorithms to remove the zigzag pattern.

I do not have a satisfied solution in current MONAI. I hope NiftiSaver can have a mode "bilinear2nearest". In this mode, the NiftiSaver will:

# In a n-label image segmentation
1. apply `argmax` to prediction, get the discreted masks.
2. apply `on_hot_encode` to the discreted masks, to get `n` channels of binary masks.
3. apply `bilinear` interpolation mode to eac channel of binary mask, get `n` channels of masks of original size.
4. apply `threshold=0.5` to all channels, get `n` channels of binary masks.
5. apply `on_hot_decode` to get the final n-label mask.

I have implementated this workflow in my own code, and it works well. I hope MONAI can also add this feature to NiftiSaver.

Note: The zigzag pattern matters especially when the interpolation factor is far from 1.

Note: this initial idea was implementated by myself these days and this idea can not remove zigzag pattern. So I followed [#3178 ], changed his Gaussian filter to Mean filter, and then got satisfied results. More details were shown bellow.

[wyli]

related to #3178

[Nic-Ma]

Hi @Jingnan-Jia ,

Thanks for your interesting experiments and detailed feedback.
If I understand correctly, is it possible to execute your operations with several transforms with interpolation modes?
For example: Resized, AsDiscreted, SaveImaged, etc.

Thanks.

[Jingnan-Jia]

Hi @wyli and @Nic-Ma , Thanks for your reply.

I have tried some experiments and found that my expectated feature is almost the same with [#3178 ].

The only difference is that the implementation of [#3178 ] is using Gaussian filger, while my implementation is using Mean filter. And my threshold is also 0.4. (By the way, my task is 3D lung lobe segmentation.)

I have tried his Gassian filter but it did not work. Maybe because I did not set the correct sigma value.

[Jingnan-Jia]

To have a better view, please click the figure and see the original size with high resolution.

[Jingnan-Jia]

The left figure is the original mask. The middle figure is the results after down-sampling and up-sampling. Obvious ziazzag pattern appeared at the edge. The right figgure is the result after I apply my smooth algorithm to the middle figure. We can see the significant improvement.

The dice between figure 1 and figure 2: 0.982
The dice between figure 1 and figure 3: 0.989

[Jingnan-Jia]

However, now I am not sure if we should add this feature to MONAI. Because it is not easy to select a good smooth hyper parameter or threshold for different masks with different resolutions.

[wyli]

closing this in favour of #3333 and #5264

NiftiSaver is also deprecated, the current preferred way is to have a postprocessing transform sequence:

https://github.com/Project-MONAI/tutorials/blob/8abcbc62c704af0bb22bedba617ae914c7802b97/modules/engines/unet_evaluation_dict.py#L89-L96

val_post_transforms = Compose(
    [
        EnsureTyped(keys="pred"),
        Activationsd(keys="pred", sigmoid=True),
        AsDiscreted(keys="pred", threshold=0.5),
        KeepLargestConnectedComponentd(keys="pred", applied_labels=[1]),
        SaveImaged(keys="pred", resample=False, output_dir="./runs/")
    ]
)