FBXLoader: wrong transform on DAZ3D models

[eriadel]

Description of the problem

Dear Ladies and Gentlemen,

We struggle with three js, DAZ3D and Mixamo, to release a new web browser game, it looks like the current blocker is in mixamo exporter or three js fbx importer (more likely).

As all mixamo's objects works with three js FBX loader we've decided to firstly export a FBX model to Mixamo from Daz Studio and then from Mixamo to Three JS to make process clear and to preserve formats compatibility.

However. In Mixamo, our FBX model works perfectly, but when we export it to Three JS from Mixamo, it looks very bad, however animation is recognized and some kind of animation is ongoing, but head, legs etc. are not in the proper place.

If you like "The Thing" from John Carpenter, then you will know what I mean ;)

Please see attachments for details 👍
https://drive.google.com/open?id=1fBJX4v7T_lQ54f8wlGPTqr-3EZjs8Q9z

Attachment contains 2 models (one imported to mixamo, one exported from mixamo) and videos showing how well model behave in mixamo and how bad it looks in three js.. :(

Thank you !

Three.js version

Latest greatest!

Browser

• All of them
• exclude internet explorer as it's not compatible with the fbx loader..

OS

• All of them

Hardware Requirements (graphics card, VR Device, ...)

none..

Best Regards,
Karol

Software Architect
www.artificialworldsltd.com

[looeee]

The model "Atlantya(exported for Mixamo).fbx" is the model exported from Daz3D right?

There are a number of warnings regarding the Euler Rotation order when I try to load that:

THREE.FBXLoader: unsupported Euler order: XZY. Currently only XYZ order is supported. Animations and rotations may be incorrect.

This is actually the first model that I've come across in the wild that has non XYZ Euler order, so I had put this fairly far down on the list of things to implement. I'll take a look, but in the meantime if it's possible, change the Euler rotation order to XYZ in Daz3d or another modelling program.

Alternatively, try using FBX2glTF to convert the final file to glTF and see if that works better.

[donmccurdy]

This has been my workflow with Mixamo https://www.donmccurdy.com/2017/11/06/creating-animated-gltf-characters-with-mixamo-and-blender/ — you can join the FBX files containing animation together in Blender, then export to glTF. I don't know whether Blender import handles other Euler orders, though.

[WestLangley]

@looeee I'd say this is a reasonable thing to try when upgrading the FBXLoader to support alternate Euler orders.

[eriadel]

Why do you guys support FBX loader, if you recommend users to convert fbx models to glTF ? ;)
(but thanks for this workaround.. right now things expand our process long enough we cancel/suspend this particular project, this could potentially generate over 300 working hours with other problems so thanks :)

DAZ Studio is quite popular software last years and I think many users will like to use it this way. (different Euler orders) Maximo does it right now, so they are step ahead of Three JS :)

Cheers! :)

[donmccurdy]

The bug report is helpful in any case, so thanks — hopefully we can fix this too. But sometimes these take a while to fix and workarounds can get you going quickly. :)

[looeee]

Fixed in #14523

[DVLP]

It looks like this problem is back. Basically every child object has wrong rotation. Top level objects are fine but if an object is in a group or has a parent it's rotation is broken.

[mrdoob]

Are you able to figure out which version broke it?

[DVLP]

I just found the commit in which it became broken.
894a79d

This is the previous commit where FBX model still looks good after importing
09e837b#diff-d6785bd34a82e05dbb309d20596bf4bf8d8e31e3c7e128fb0917fdebe36a9006

Locally I was able to see some improvement after restoring the old "setModelTransports" function but that only fixes the problem partially.

[DVLP]

Daz model. In original only one hand is in T-pose, the other is pointing down. I just bought this model and making first experiments.

Before commit(correct pose, irrelevant bugs) 894a79d:

After commit(horror!) 894a79d:

After restoring "setModelTransport" and removing
`sceneGraph.traverse( function ( node ) {

			if ( node.userData.transformData ) {

				if ( node.parent ) node.userData.transformData.parentMatrixWorld = node.parent.matrix;

				var transform = generateTransform( node.userData.transformData );

				// if ( node.parent ) transform = node.parent.matrix.getInverse( node.parent.matrix ).multiply( transform );

				node.applyMatrix( transform );

				// if ( node.parent )


			}

		} );`

in the latest FBX loader
(not fully correct pose but better)

[mrdoob]

Thanks for finding the commit!
Unfortunately, FBXLoader doesn't seem to have an active maintainer...

[DVLP]

I'll try to do some experiments, hopefully I can fix this with trial and error because I'm not an expert in either FBX spec or the maths behind.

[mrdoob]

Would converting the FBX to GLTF work?

https://github.com/facebookincubator/FBX2glTF
https://github.com/facebookincubator/FBX2glTF/releases/download/v0.9.7/FBX2glTF-windows-x64.exe
https://github.com/facebookincubator/FBX2glTF/releases/download/v0.9.7/FBX2glTF-darwin-x64

[DVLP]

probably yes but I prefer FBX format for easy export from 3ds max without plugins to my game. It uses GLTF(actually GLB) but after importing. https://editor.bad.city

[DVLP]

Ok so setModelTransport and getTransportData functions are irrelevant.
Only replacing "generateTransform" with the old one fixes everything but it looks like it's missing half of the functionality.

Correct pose in the latest FBXLoader with generateTransform taken from 09e837b

[DVLP]

The whole thing is broken and it only works because it malfunctions.

This function is derived from a function called CalculateGlobalTransform and as the name suggest it's for getting global transformations. But if I understand correctly it's used here to extract local transformations. So why does it even work?

It only works because it's broken. The line below actually wipes the parent matrix lParentGX and overwrites it with just defiend, empty lParentGRM
lParentGX.extractRotation( lParentGRM );
compared to the source code in Autodesk docs it should be
lParentGRM.extractRotation( lParentGX );

Both are now empty matrices. After setting this to the correct order everything turns into a total mess. That's because global transforms applied as local now work and it's all broken.

I was thinking about reverting to the old function but this function seems to add support for pivots. So instead I found that adding lPostRotationM inversion like in the old generateTransforms function, while retaining that malfunctioning(empty) lParentGX fixes the whole thing

lPostRotationM.getInverse(lPostRotationM);

above this line
// Pivots and offsets

[looeee]

This function is derived from a function called CalculateGlobalTransform and as the name suggest it's for getting global transformations. But if I understand correctly it's used here to extract local transformations

It's entirely possible that this function is being wrongly applied. I would welcome additional eyes on this as it's a complex part of the code and I don't think it's been examined by anyone except for me when I wrote it. If you create a PR with a fix I'll review it and test against the collection of FBX models that I've gathered while working on the loader.

[DVLP]

There are a few changes to make it closer to Autodesk example. I don't know why post-rotation inversion is required but that's how it worked before the breaking change and without it it doesn't work properly.

After fixing extractRotation everything became broken because this function computes global transformation so at the end I added premultiplying by inverted parent world transformation matrix and it seems to work. Another change is using parent world matrix instead of it's matrix but then the last method of inheritance is using parent's local scaling so ended up using both matrix and matrixWorld

Here's the PR
#20932

[DVLP]

@looeee my Daz model now looks good but in another FBX file elements with parents still don't look correct. Actually even with the old generateTransform they don't look correct so the mystery continues. I may be wrong because there's a lot of going on after importing FBX in my modified editor. I think this can be merged and the issue closed because it fixes skeletons but there's a new issue tracking needed for scene elements with parents.

[DVLP]

After doing some experiments I can see that child elements are now behaving as expected. The new problem I mentioned is non-existent. It's a malfunction of my own function I use to bake scaling so imported objects have their scale reset to 1.