Better anti-aliasing with shaders

[lhofhansl]

Problem

I notice some annoying moire patterns, especially with blocks fairly far away - like 300-400+

Note that these are due to mapnode boundaries that are rendered with a higher frequency then the screen can render. FXAA does not help with that, and SSAA is way to too expensive - at least on my NVidia card.

This really needs a geometry interpolation. The "hardcoded" FSAA (also called MSAA) works really well, without a lot of performance issues, but that does not work with MT's shaders.

So I think we need the equivalent of FSAA as a shader.

Solutions

Implement MSAA as Shader

Alternatives

Improve SSAA performance - not sure that's possible as it needs to render the entire scene with a higher resolution, eating up a lot of resources even just with a setting of 2 (which requires 4x as many texels/pixels to be rendered).
Or change FXAA (somehow) to deal with this scenario (update, this cannot be done as a pixel shader, it's too late by then).

Additional context

If you want to try it out, set a large viewing_range and observer block in a distance (500 nodes away or so), especially snow or sand.
Notice that FSAA deals with quite well, that FXAA does not help at all, and SSAA help a little with a setting of 2 but gets much slower.

[HybridDog]

There exists a technique called rotated-grid super sampling, where, if I understand it correctly, the textures and not the whole image is super-sampled.

Improved Alpha Testing Using Hashed Sampling explains a technique which fixes the problem where textures which have fully transparent or opaque textures look bad in the distance, especially with enabled mip mapping. Issue #6846 is about this problem. The technique described in that paper can probably be adopted/generalised to nearest-neighbour texture sampling, which could prevent Moire artifacts in Minetest.

Temporal anti-aliasing is an alternative to SSAA. It averages the pixels over time instead of spatially. It may also be compatible with the sharpness-preserving algorithm which I have used for SSAA in #13959.

Anisotropic filtering with nearest-neighbour interpolation works very well against Moire patterns in my opinion. However, only some GPUs support it. To support more hardware, anisotropic filtering could be implemented in a shader, but I don't know how this would affect performance.

A long time ago in #9247, I have changed shaders a bit to fix multisample anti-aliasing MSAA problems. Why does MSAA no longer work if shaders are enabled?

By the way: Anti-aliasing, anisotropic filtering, mip mapping, and similar techniques currently work incorrectly in Minetest because the techniques are applied on non-linear sRGB colours.

[lhofhansl]

Why does MSAA no longer work if shaders are enabled?

I had just assume MSAA is incompatible with shaders. So good question!

Commit that added the comment about the incompatibility: c09a3a5
Commit that added pipelining: ff6dcfe

[rollerozxa]

MSAA stopped working with shaders enabled after the rendering pipeline was merged I believe.

[lhofhansl]

MSAA stopped working with shaders enabled after the rendering pipeline was merged I believe.

Does anybody understand why this is the case? Because we cannot pass fragment-samples between stages?

Update: I guess it'd require use of sampler2Dms instead of sampler2D and the appropriate binding on the host side.

Update II: If anybody know how to combine this with MSAA, please me know. Something around avoiding resolving the MS texture until later. I do not understand how that would work with multiple stages with each having its own vertex shader. (Searching around for this just reveals that many folks are struggling with that)
I.e. I want to take the geometry based super sampling at least into the first stage (before post processing).

[lhofhansl]

In case anyone is interested. If I remove the postprocessing logic:

diff --git a/src/client/render/plain.cpp b/src/client/render/plain.cpp
index dccdaedb3..17cb5cc5e 100644
--- a/src/client/render/plain.cpp
+++ b/src/client/render/plain.cpp
@@ -110,8 +110,8 @@ std::unique_ptr<RenderStep> create3DStage(Client *client, v2f scale)
RenderPipeline *pipeline = new RenderPipeline();
pipeline->addStep(pipeline->own(std::unique_ptr<RenderStep>(step)));

-               auto effect = addPostProcessing(pipeline, step, scale, client);
-               effect->setRenderTarget(pipeline->getOutput());
+               //auto effect = addPostProcessing(pipeline, step, scale, client);
+               //effect->setRenderTarget(pipeline->getOutput());
step = pipeline;
}
return std::unique_ptr<RenderStep>(step);

MSAA/FSAA works as expected.

[lhofhansl]

The difference can be quite dramatic:

With MSAA (disregard the unloaded blocks - I just did not wait long enough, and of course it's not doing tonemapping now):

No AA, or FXAA (FXAA does not help at all in this case):

And it is much worse when you move around as then the Moire pattern moves as well. So it seems important to find a a way to fix this.

[lhofhansl]

I'll create a PR today that allow disabling post-processing. That way it's easy to test.
Note that MSAA is not incompatible with shaders, but incompatible with PP (at least as done in MT).

In the end, I think that maybe we merge the postprocessing pipeline to eagerly. Not sure how proceed in general now. We still want at least tonemapping to happen without the postprocessing pipeline.

[lhofhansl]

@GefullteTaubenbrust2 FYI, in case you feel inclined to look at this.