Add udimextract and udimtexcoord nodes

[crydalch]

These are two new nodes which let users commandeer UDIM support, to drive texture variation.

• udimtexcoord - provides the texture coordinates of the specified UDIM patch. The UDIM patch can be driven by geometry properties, letting users drive texture maps using primvars.
• udimextract - extracts the UDIM patch from the given texture coordinates.

MaterialXViewer and MaterialXGraphEditor both don't recognize image paths with the <UDIM> token, but here is a render from Karma. Each rubber toy is assigned a random integer primvar between 1001 and 1025.

[crydalch]

While these nodes are related to the same artistic needs motivating issue #878, they are not intended to be a complete solution. What this does provide is a very simple way for artists to drive texture variation with primvars. Other than UDIM support, no additional work is needed on the part of the renderer or MaterialX.

Currently these nodes limit the UDIMs to 1001-1099 (inclusive), which is the range of UDIMs that USD supports. However if UDIM ever becomes an actual specification, we can adjust the limits accordingly.

[jstone-lucasfilm]

@crydalch How does this proposal relate to the existing support for <UDIM> tokens in MaterialX and USD/MaterialX?

https://github.com/AcademySoftwareFoundation/MaterialX/blob/main/documents/Specification/MaterialX.Specification.md#filename-substitutions

These tokens currently work as expected in MaterialXView, for example, and I've successfully tested multi-UDIM MaterialX files in Houdini/Karma as well.

As far as I can tell, Houdini/Karma correctly integrates the native UDIM support in MaterialX, and will assign each UDIM texture to the associated geometric UDIM at render time:

https://www.sidefx.com/docs/houdini/solaris/materialx.html#textures

[crydalch]

Thanks @jstone-lucasfilm! Yeah this isn’t trying to “fix” UDIM support. These nodes allow users to leverage existing UDIM support, to drive texture variation using primvars. Full string/primvar token support (i.e. what’s been discussed in #878) is going to require more work on the part of MaterialX, UsdMaterialX, USD, and render delegates. While eventually that would likely be the more flexible approach, this seemed like a good alternative, which requires nothing beyond UDIM support. So even when primvar-substitution lands, (ab)using UDIM textures like this proves useful.

Maybe changing the names would help clarify things. Instead of udimtexcoord we could use udimoffset? Or udimextract could be udimextractpatch?

BTW, I actually haven’t been able to get a <UDIM> texture work in MaterialXViewer or MaterialXGraphEditor. I just get a message saying Image file not found: /home/chrisr/Desktop/udimgrids/uvgrid.<UDIM>.jpg

Maybe something with my local build is the culprit? Or is there something messed up with this dataset? The bunny works in Houdini just fine. Does MaterialX have example geometry/texture resources for UDIMs?

Here is the .obj:

bunny.zip

Thanks!

[jstone-lucasfilm]

In MaterialX, you just need to state the udimset for a multi-UDIM material, so that it can be rendered and run through material processes (e.g. shader translation, texture baking) in tools that don't have access to the geometry as well.

Here's a multi-UDIM version of the bunny, with the corresponding render in MaterialXView:

MultiUdimBunny.zip

I like the idea of including a multi-UDIM example asset in the MaterialX repository, and let me know if you have ideas for one!

[crydalch]

Ah - it was me :) Thank you for that example, I see it working now in MaterialX tools!

Just to clarify the usage a bit better, I've attached image shows some usage examples. The shader has udimtexcoord connected to an image node. Images 1-3 are manually setting the UDIM patch on udimtexcoord, while the 4th image is reading the udim number from a primvar on each instance.

Maybe we can chat about this a the next TSC meeting? I'm still wrapping my head around the whole udimset concept. However it's possible that how these nodes (ab)use UDIM support for primvar-driven texture variation can't work in a pure MaterialX environment.

[jstone-lucasfilm]

@crydalch Sure, let's plan to pick up this topic in our upcoming MaterialX TSC meeting.

I think the difference you're encountering may not be between MaterialX and USD/MaterialX workflows (which have nearly identical interpretations of UDIMs), but between real-time and off-line renderers (e.g. Storm vs. Karma).

Real-time renderers traditionally require all textures referenced by an asset to be loaded in advance of shader execution, while off-line renderers can make this decision dynamically at shade time. This makes it much more challenging for a real-time renderer to respond to shader-computed UDIM indices by loading a new set of textures from disk, and it's common for this category of renderer to disallow this approach.

[kwokcb]

Adding some small notes based thoughts from the TSC. This is in the context that a texture atlas is used for real-time:

1. I think you still need to have the full udim set available otherwise the atlas is unbounded.
2. There are utilities to remap to a normalized atlas which pulled from Maya H/W rendering and are now found in core utilities and used by the h/w shader generator image node (tiled image can work if the atlas is normalized I think).
   But a normalized atlas still needs requires a udim set.
3. In Storm I'd assume the hierarchical tiling system also needs pre-knowledge of the udim set (range).
4. Adding the onus of something like the Unreal 2-pass system to id patches for resource loading sounds too complicated for something like the sample viewer.

So I think as long as the udim lookup does not go outside the udim set per image then it's something that can work theoretically if you have normalized udim atlas (static or dynamic). BTW, I've always found the MaterialX udim set specification is a bit "odd" in that it's Document level however -- but these nodes are for a specific image, so I think the new wrinkle is that this adds a different "scope" of udim evaluation.

[jstone-lucasfilm]

Just to align our two discussions of this subject, I'll forward a high-level summary from today's MaterialX TSC meeting, where we had the chance to discuss this topic a bit:

Three common approaches for handling UDIMs in rasterization engines are:

1. Partition the geometry based on its texture coordinates, and assign each UDIM to its corresponding partition (e.g. MaterialXView)
2. Combine all possible UDIMs into a texture array, and select which UDIM is rendered at shade time (e.g. Storm)
3. Dynamically load textures into memory using Sparse Virtual Texturing (e.g. Mari, UE5)

Approaches (1) and (2) are relatively limited, and either place a cap on UDIM count or prevent dynamic UDIM offsets.
Approach (3) potentially allows both arbitrary UDIM counts and dynamic UDIM offsets.

My sense is that approach (3) is the ideal, but a more practical alternative would be for MaterialX to support both (1) and (2), allowing clients of the library to choose which one they prefer. This would allow applications such as Storm to continue using pre-generated texture arrays, and would allow MaterialXView to switch between these two approaches dynamically as needed by users.

[jstone-lucasfilm]

In a side discussion with @crydalch, we resolved that it makes more sense to work with the USD/MaterialX teams to resolve the core issue in #878, so I'm going to close this pull request for now.

There's a USD/MaterialX working group within the ASWF that focuses on these alignment issues, and this subject would be a great one to bring up at their next meeting.