alembicReader.cpp uses fixed frame rate of 24 FPS

[sybrenstuvel]

Description of Issue

When loading an Alembic file via the Alembic-USD plugin it is assumed to be 24 FPS.

This can be seen in alembicReader.cpp lines 898 and 961. The Alembic file contains timing information, and this should be used instead of hard-coding the frame rate.

Note that this limitation is not documented in the Known Limitations section of the Alembic-USD plugin documentation.

Steps to Reproduce

Here is a zipped Alembic file: bug_rotation_scale_alembic.zip. This file contains 20 frames at a frame rate of 30 FPS. Use abcls -t bug_rotation_scale_alembic.abc to show the timing information. There are two time sample series in there, one for geometry (there is one sample, as the geometry itself is not animated), and one for transforms (20 samples @ 30 FPS):

% abcls -t bug_rotation_scale_alembic.abc 

Time Samplings: 
0 Uniform Sampling. Start time: 0 Time per cycle: 1
Max Num Samples: 1
1 Uniform Sampling. Start time: 0.0333333 Time per cycle: 0.0333333
Max Num Samples: 20

Reproduction method 1

1. Run usdview bug_rotation_scale_alembic.abc
2. Play the animation
3. See the disc change scale. I suspect this happens because of incorrect linear per-component interpolating of transformation matrices.

Reproduction method 2

1. Run usdcat bug_rotation_scale_alembic.abc
2. See that it contains samples for frames 0.8 to 16, which is a factor of 24/30 off.

Also, usdcat mentions endTimeCode = 456 in the USD metadata, even though the Alembic file only contains 20 frames. I'm not sure whether this is caused by the same bug or not. What I do know is that this number increases when more than one time sample per frame is written to Alembic.

System Information (OS, Hardware)

Linux, Kubuntu 19.04, Intel Core i7 64-bit.

Package Versions

USD 19.07, self-built from the Git repository v19.07 tag.

Build Flags

Built using:

python2 build_scripts/build_usd.py /opt/usd-with-python --generator Ninja --alembic

[spiffmon]

Hi Sybren! This behavior is not documented as a abcReader limitation because it is an Alembic limitation. I confirmed awhile back on the Alembic forum that Alembic has no official means to encode an overall samples/timeCodes/frames per-second. abcls is only telling you how the individual properties are sampled. If the scene contained a non-linearly deforming Mesh, a user may have decided to export five or ten samples per frame for just that mesh’s points property - we do this all the time when exporting USD from Presto. In that case, how is abcReader supposed to figure out which of the three timeSamplings it should use to recover timeCodesPerSecond? I was surprised how unconcerned the abc/usd-using community was about this when we asked about it... there was a bug in abcReader that went unreported for almost three years where we double-compensated the samples, so that animation played back at 1/24th speed! It’s been fixed now for awhile, though. We would be happy to accept a PR that improves the translation in a way that’s acceptable to the community- one idea is that since a studio’s FPS doesn’t usually vary from shot to shot, one could add an env var to specify the desired timeCodesPerSecond. One could also add a SdfFileFormatArg to the plugin so that each reference to an abc file could specify what it wants. Or maybe the heuristic you’re relying on covers a high percentage of cases? I think there’d still need to be a way to override it, though. You are right about the transformation handling; another pr we’d be happy to accept is one that translates abc xformOps into usdGeom xformOps, instead of doing the simple-to-implement flattening we are currently doing.

On Fri, Aug 16, 2019 at 4:58 AM Sybren A. Stüvel ***@***.***> wrote: Description of Issue When loading an Alembic file via the Alembic-USD plugin it is assumed to be 24 FPS. This can be seen in alembicReader.cpp lines 898 <https://github.com/PixarAnimationStudios/USD/blob/32ca7df94c83ae19e6fd38f7928d07f0e4cf5040/pxr/usd/plugin/usdAbc/alembicReader.cpp#L898> and 961 <https://github.com/PixarAnimationStudios/USD/blob/32ca7df94c83ae19e6fd38f7928d07f0e4cf5040/pxr/usd/plugin/usdAbc/alembicReader.cpp#L961>. The Alembic file contains timing information, and this should be used instead of hard-coding the frame rate. Note that *this limitation is not documented* in the Known Limitations section of the Alembic-USD plugin documentation <https://graphics.pixar.com/usd/docs/Alembic-USD-Plugin.html>. Steps to Reproduce Here is a zipped Alembic file: bug_rotation_scale_alembic.zip <https://github.com/PixarAnimationStudios/USD/files/3509430/bug_rotation_scale_alembic.zip>. This file contains 20 frames at a frame rate of 30 FPS. Use abcls -t bug_rotation_scale_alembic.abc to show the timing information. There are two time sample series in there, one for geometry (there is one sample, as the geometry itself is not animated), and one for transforms (20 samples @ 30 FPS): % abcls -t bug_rotation_scale_alembic.abc Time Samplings: 0 Uniform Sampling. Start time: 0 Time per cycle: 1 Max Num Samples: 1 1 Uniform Sampling. Start time: 0.0333333 Time per cycle: 0.0333333 Max Num Samples: 20 Reproduction method 1 1. Run usdview bug_rotation_scale_alembic.abc 2. Play the animation 3. See the disc change scale. I suspect this happens because of incorrect linear per-component interpolating of transformation matrices. Reproduction method 2 1. Run usdcat bug_rotation_scale_alembic.abc 2. See that it contains samples for frames 0.8 to 16, which is a factor of 24/30 off. Also, usdcat mentions endTimeCode = 456 in the USD metadata, even though the Alembic file only contains 20 frames. I'm not sure whether this is caused by the same bug or not. What I do know is that this number increases when more than one time sample per frame is written to Alembic. System Information (OS, Hardware) Linux, Kubuntu 19.04, Intel Core i7 64-bit. Package Versions USD 19.07, self-built from the Git repository v19.07 tag. Build Flags Built using: python2 build_scripts/build_usd.py /opt/usd-with-python --generator Ninja --alembic — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#940>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABPOU2GDYRD6HHIB4OOMUB3QE2I5NANCNFSM4IMGZ4HA> .

-- --spiffiPhone

[sybrenstuvel]

Hi Sybren! This behavior is not documented as a abcReader limitation because it is an Alembic limitation. I confirmed awhile back on the Alembic forum that Alembic has no official means to encode an overall samples/timeCodes/frames per-second.

Sure, Alembic can store samples at arbitrary timecodes. This is not a compelling reason to resort to hard-coding 24 FPS, though. With a scene in a fixed frame rate, there is a clear mapping from frame number to timecode, and Alembic provides a well-defined way to obtain samples given that timecode. We do this all the time in Blender, and we also support sub-frame sampling. I don't see why this can't be done with USD.

how is abcReader supposed to figure out which of the three timeSamplings it should use to recover timeCodesPerSecond?

All the geometry schemas in Alembic have a getTimeSampling() function that you can use to determine the time sampling.

Maybe the heuristic you’re relying on covers a high percentage of cases?

We're not relying on any heuristic, we're just using those getTimeSampling() functions. See the get_matrix() function in Blender's Alembic code for an example.

[spiffmon]

I understand that any schema or property in Alembic can report me its timeSampling - the problem is that Alembic restricts itself to talking about samples per second, and not frames per second, and as far as I know, no mapping of one to the other. It’s a subtle distinction, but important, which I think we can demonstrate with a simple example. Time Samplings: 0 Uniform Sampling. Start time: 0 Time per cycle: 1 Max Num Samples: 1 1 Uniform Sampling. Start time: 0.016666666666667 Time per cycle: 0.016666666666667 Max Num Samples: 20 So should we interpret that as 60fps with samples on 1’s, or as 30fps with samples on 0.5’s? Each seems equally likely and equally valid, but would produce a different layout of samples in USD, and a different value of timeCodesPerSecond. Is that analysis incorrect or lacking?

--spiff

On Fri, Aug 16, 2019 at 8:09 AM Sybren A. Stüvel ***@***.***> wrote: Hi Sybren! This behavior is not documented as a abcReader limitation because it is an Alembic limitation. I confirmed awhile back on the Alembic forum that Alembic has no official means to encode an overall samples/timeCodes/frames per-second. Sure, Alembic can store samples at arbitrary timecodes. This is not a compelling reason to resort to hard-coding 24 FPS, though. With a scene in a fixed frame rate, there is a clear mapping from frame number to timecode, and Alembic provides a well-defined way to obtain samples given that timecode. We do this all the time in Blender, and we also support sub-frame sampling. I don't see why this can't be done with USD. how is abcReader supposed to figure out which of the three timeSamplings it should use to recover timeCodesPerSecond? All the geometry schemas in Alembic have a getTimeSampling() function that you can use to determine the time sampling. Maybe the heuristic you’re relying on covers a high percentage of cases? We're not relying on any heuristic, we're just using those getTimeSampling() functions. See the get_matrix() function <https://developer.blender.org/diffusion/B/browse/master/source/blender/alembic/intern/abc_object.cc$223> in Blender's Alembic code for an example. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#940>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABPOU2GWFPDJLSIZLXN5DDLQE27KBANCNFSM4IMGZ4HA> .

-- --spiffiPhone

[jilliene]

Filed as internal issue #USD-5497

[sybrenstuvel]

I think there are a few things going on at the same time:

1. USD files declare a fixed frame rate and ABC files do not, so when converting ABC → USD this needs to be guestimated. I understand that this is hard given the difference in time encoding in these formats.
2. The Alembic-USD plugin assumes the ABC file is a hard-coded 24 FPS. This is not documented in the "Known Limitations" of the Alembic-USD plugin because it is considered to be a known limitation of Alembic (and not the plugin). However, the hard-coding happens in that plugin, so it's a limitation of that plugin. Alembic files are by no means limited to this frame rate.
3. Because of the assumption that the file is 24 FPS, samples in ABC don't align with frames in USD. This in turn requires interpolation between samples, which, for transformation matrices, is done with component-wise LERP. It is well known that LERPing such matrices produces erroneous results.
4. When converting an ABC file with inter-frame samples to USD, the reported endTimeCode in the USD file is incorrect. Regardless of the above, I think you'll agree that this is a bug. After all, even when the last frame is written as 16 (a factor of 24/30 off), the endTimeCode should be 16.

So should we interpret that as 60fps with samples on 1’s, or as 30fps with samples on 0.5’s?

I think both are an improvement over the hard-coded 24 FPS, as either interpretation will have a sample in ABC exactly on a frame in USD. This works around point 3 from the list above. It also improves playback performance as no interpolation needs to be performed.

The root cause point 1 above is, I suspect, that Alembic files refer to scene time, whereas the conversion to USD happens without any reference to any existing scene. Maybe the USD and Alembic developers+community can agree on some metadata fields in Alembic that expose the scene frame rate at time of exporting?

[spiffmon]

I am with you 100%! I’m not sure what’s up with #4... there was a thing where there was a convention with some alembic exporters to put single/unvarying samples at some time sample outside the animation frame range, but I thought it was really early rather than really late... again, since alembic files don’t officially declare the start and end of the “useful” animation, we need to guess, by unioning all the timesamples. I don’t know if it would go against alembic design decisions, but would you be willing to pose your closing question to the alembic google group, referencing this issue? We’d be very happy to incorporate such a field into our reader; there are a lot of alembic exporters that would need to be updated, but hitting maya and blender would be a good start. Cheers, —spiff

On Tue, Aug 20, 2019 at 2:19 AM Sybren A. Stüvel ***@***.***> wrote: I think there are a few things going on at the same time: 1. USD files declare a fixed frame rate and ABC files do not, so when converting ABC → USD this needs to be guestimated. I understand that this is hard given the difference in time encoding in these formats. 2. The Alembic-USD plugin assumes the ABC file is a hard-coded 24 FPS. This is not documented in the "Known Limitations" of the Alembic-USD plugin because it is considered to be a known limitation of Alembic (and not the plugin). However, the hard-coding happens in that plugin, so it's a limitation of that plugin. Alembic files are by no means limited to this frame rate. 3. Because of the assumption that the file is 24 FPS, samples in ABC don't align with frames in USD. This in turn requires interpolation between samples, which, for transformation matrices, is done with component-wise LERP. It is well known that LERPing such matrices produces erroneous results. 4. When converting an ABC file with inter-frame samples to USD, the reported endTimeCode in the USD file is incorrect. Regardless of the above, I think you'll agree that this is a bug. After all, even when the last frame is written as 16 (a factor of 24/30 off), the endTimeCode should be 16. So should we interpret that as 60fps with samples on 1’s, or as 30fps with samples on 0.5’s? I think both are an improvement over the hard-coded 24 FPS, as either interpretation will have a sample in ABC exactly on a frame in USD. This works around point 3 from the list above. It also improves playback performance as no interpolation needs to be performed. The root cause point 1 above is, I suspect, that Alembic files refer to scene time, whereas the conversion to USD happens without any reference to any existing scene. Maybe the USD and Alembic developers+community can agree on some metadata fields in Alembic that expose the scene frame rate at time of exporting? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#940>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABPOU2GHVKBONENVGTJZOUDQFOZLXANCNFSM4IMGZ4HA> .

-- --spiffiPhone

[sybrenstuvel]

but would you be willing to pose your closing question to the alembic google group, referencing this issue?

Done: https://groups.google.com/forum/#!topic/alembic-discussion/f0P3bKNe_qQ

[Narann]

I realize all I said in my answer has already been discussed here, sorry for this. 🙏

[spiffmon]

Hi @sybrenstuvel ,
It doesn't seem like the query on the Alembic forum went anywhere? I'd like to suggest an alternative, which is to start using the timeCodesPerSecond metadata that our Alembic reader/writer already uses to be able to roundtrip USD files through Alembic. The Alembic files Blender produces will then just have this extra metadata that only Blender and USD will understand, but that may be a pretty good starting point.

Cheers,
@spiffmon

[sybrenstuvel]

@spiffmon It actually was implemented and went into the 1.7.12 release, it just wasn't communicated to the Github issue tracker. I'm now modifying Blender to link against the new Alembic release and write this new metadata field.

[spiffmon]

I saw it in the release notes, @sybrenstuvel ! I created Issue #1023 with details on how usdAbc might take advantage of it. We’d be thrilled if you were able to tackle that project.

[boberfly]

Hi, I think this issue can now be closed as of #2944 right? I was shocked to run into this but noticed it has been fixed.

[spiffmon]

Indeed - thanks, @boberfly !