Port Shadow Improvements from gazebo11

[scpeters]

Desired behavior

There were numerous shadow improvements added to gazebo11 in bitbucket pull request 2805 (gazebosim/gazebo-classic@f6612bf), including the CustomPSSMShadowCameraSetup class, that have not yet been ported to gz-rendering. We should any useful missing features.

Alternatives considered

Implementation suggestion

Additional context

[darksylinc]

Hi!

I just saw your ticket. I can make a few comments / add info regarding OgreNext (aka ogre2 render engine, not ogre1):

CustomPSSMShadowCameraSetup

As far as I understood, gazebo11 creates its own CustomPSSMShadowCameraSetup to override the LiPSM warping.

OgreNext did the same thing, and in fact rewrote the FocusedShadowCameraSetup to be more accurate (although not necessarily higher quality). There were various bugs that we got rid of.

HW PCF

This is on by default.

Multiple PCF quality options

I'm looking for setShadowSettings in the codebase but I couldn't find any hits. Which must mean it's using the 3x3 PCF default.

OgreNext offers more quality options:

/// Standard quality. Very fast.
PCF_2x2,

/// Good quality. Still quite fast on most modern hardware.
PCF_3x3,

/// High quality. Very slow in old hardware (i.e. DX10 level hw and below)
/// Use RSC_TEXTURE_GATHER to check whether it will be slow or not.
PCF_4x4,

/// Better and slower than 4x4, same considerations
PCF_5x5,

/// Better and slower than 5x5, same considerations
PCF_6x6,

/// High quality. Produces soft shadows. It's much more expensive but given
/// its blurry results, you can reduce resolution and/or use less PSSM splits
/// which gives you very competing performance with great results.
/// ESM stands for Exponential Shadow Maps.
ExponentialShadowMaps,

Regarding ESM, it's a hit-or-miss (which is why it's not the default). It works really great indoors but requires a lot of care and tuning depending scene conditions, which is why it's not the default.

Poisson sampling

There is no out of the box support in OgreNext but a user contributed code.

We can incorporate that code into both OgreNext and gazebo if the option looks appealing.

Texture resolution

This is in Ogre2Scene::UpdateShadowNode and Ogre2Scene::CreateShadowNodeWithSettings in ogre2/src/Ogre2Scene.cc

That's where you'll find texture resolution and settings like number of PSSM splits, PSSM lambda and split blend options.

Gazebo never touches ShadowTextureDefinition::numStableSplits but it is also an available option. This setting trades off a bit of shadow quality on the the first N splits (N = numStableSplits) in order to have stable shadow maps. The quality tradeoff is too large for the last splits, which is why the last splits should often not use it.

What are stable shadow maps? With out of the box PSSM, when you rotate the camera, the shadows will "swim" and move as the shadow map camera is analyzed to best fit the current camera and aliasing artifacts appear.

Although technically each individual frame is getting the best possible quality, the effect in motion can be quite distracting.

This video shows what I'm talking about:

out.mp4

Hybrid research

Back in 2016 I had a working prototype of an hybrid raytracer/rasterizer approach to shadow mapping that was fast and produced sharp shadows with no shadow map aliasing (only "regular aliasing" like normal geometry enabled without MSAA enabled) and a few weeks ago I learnt of a trick that would fix a major issue that was making it impractical (massive memory consumption).

However:

• It's technically R&D. It'd be a novel technique.
  • Results are not guaranteed
• It could consume a lot of memory ("a lot" is relative)
• Geometry needs a preprocessing step
  • We already have this in Ogre::Mesh::msOptimizeForShadowMapping which is true when using our offline tools to generate meshes; but false at runtime.
  • If you need to generate a lot of geometry dynamically every frame, it may be a problem

If the available options are not good enough; we can discuss this option.

[mogumbo]

If memory serves, CustomPSSMShadowCameraSetup was created to replace LisPSM with FocusedShadows (as you mentioned) and also to recompute frusta using z-up math instead of y-up math. The up-vector conflict between Gazebo and Ogre was causing the shadow maps to spin and stretch on the ground (shadows may have been technically correct but the resolution was all over the place). If OgreNext is still y-up then we can probably expect the same problems.

The hybrid research sounds fun. Is it easy to use those shadow maps if you are using custom shaders?

In gz-rendering will it be possible to have PSSM with 4 maps instead of 3? I always thought that would have made tuning shadows a lot easier.

[darksylinc]

and also to recompute frusta using z-up math instead of y-up math. The up-vector conflict between Gazebo and Ogre was causing the shadow maps to spin and stretch on the ground

AFAIK there is nothing in the new code that is y-up specific so it should work (and there are other projects using z-up + OgreNext).

However I wouldn't be surprised if the math inadvertently assumes y-up somewhere by mistake.

The hybrid research sounds fun. Is it easy to use those shadow maps if you are using custom shaders?

If by 'custom shaders' you mean low level materials; then it might work but probably no.
If by 'custom shaders' you mean using Hlms customizations then yes.

Since you come from Ogre1; in OgreNext Hlms ("High Level Material System") is the main system that generates shaders automatically and drives most of the pipeline.

Although you can still use the old "low level" materials (i.e. Ogre1's Materials), for both in terms of performance and harmony of features it's better to use Hlms customizations.

Hlms has a system for asking users to provide their own "pieces" of code that will be inserted into various parts of the autogenerated shader.

That way entire portions can be overriden to perform what you want while benefiting from the rest of the code being autogenerated.

gz-rendering in fact was until recently using low level materials for overriding rendering for the Thermal and Lidar cameras, but had a few bugs like Skeletal animations not working. In #545 these shaders were migrated to become Hlms customizations to fix those issues.

In gz-rendering will it be possible to have PSSM with 4 maps instead of 3? I always thought that would have made tuning shadows a lot easier.

Yes.

[mogumbo]

The hybrid research sounds fun. Is it easy to use those shadow maps if you are using custom shaders?

If by 'custom shaders' you mean low level materials; then it might work but probably no. If by 'custom shaders' you mean using Hlms customizations then yes.

I mean vertex and fragment shader files from scratch. So that sounds like a "probably not."