DASH trick mode tracks

[joeyparrish]

Shaka Player has no knowledge of DASH trick mode tracks. According to DASH IOP v3.3, trick mode tracks should not be selected for normal playback.

Steps for us:

1. find sample assets for trick mode so that we can properly test this feature
2. ignore trick mode tracks, since we do not yet support them (we should get at least to this stage as soon as possible, for release in v2.1 or v2.0.1)
3. use trick mode tracks when player.trickPlay() is used, limited to maxPlayoutRate from the manifest (this may take much longer to achieve, and may or may not make v2.1)

Here's the relevant text from IOP v3.3:

3.2.9. Trick Mode Support

Trick Modes are used by DASH clients in order to support fast forward, seek, rewind and other operations in which typically the media, especially video, is displayed in a speed other than the normal playout speed. In order to support such operations, it is recommended that the content author adds Representations at lower frame rates in order to support faster playout with the same decoding and rendering capabilities.

However, Representations targeted for trick modes are typically not be suitable for regular playout. If the content author wants to explicitly signal that a Representation is only suitable for trick mode cases, but not for regular playout, it the following is recommended:

• add an Adaptation Set that that only contains trick modes Representations
• annotate the Adaptation Set with an EssentialProperty descriptor or SupplementalProperty descriptor with URL "http://dashif.org/guidelines/trickmode" and the @value the value of @id attribute of the Adaptation Set to which these trick mode Representations belong. The trick mode Representations must be time-aligned with the Representations in the main Adaptation Set. The value may also be a white-space separated list of @id values. In this case the trick mode Adaptation Set is associated to all Adaptation Sets with the values of the @id.
• signal the playout capabilities with the attribute @maxPlayoutRate for each Representation in order to indicate the accelerated playout that is enabled by the signaled codec profile and level.

If an Adaptation Set is annotated with the EssentialProperty descriptor with URI "http://dashif.org/guidelines/trickmode" then the DASH client shall not select any of the contained Representations for regular playout.

[joeyparrish]

@maxPlayoutRate is not the only limiting factor for trick play rate. We should also make sure that the trick play track's bandwidth, multiplied by the trick play rate, does not exceed the user's estimated bandwidth.

For example, in this test clip from the DASH IF, the lowest bitrate normal stream is about 3x the bitrate of the trick play stream. For clients unable to play any but the lowest bitrate stream, the max trick play rate would be about 3.

Another way to look at it, which would still work for content without a trick play track, is to consider the current playback rate in track selection. Choose a track based on estimated bandwidth divided by current playback rate.

[joeyparrish]

Two working test clips from the DASH IF:

• http://dash.akamaized.net/dash264/TestCases/9b/qualcomm/1/MultiRate.mpd
• http://dash.akamaized.net/dash264/TestCases/9b/qualcomm/2/MultiRate.mpd

[joeyparrish]

Finally, we should consider how reverse trick play works. Today, browsers do not support negative playback rates. It is unclear if they ever will. We currently use repeated seeking to simulate negative playback rates, but this is not a great experience.

Our forward trick play experience will be greatly improved by the use of special trick mode tracks, so we should look for a way to improve our reverse trick play experience as well. Perhaps having a special trick mode track will open up another possibility or allow us to refine the technique we use today.

[joeyparrish]

The above-referenced trick-play assets fail to decode in Chrome (except the trick play tracks themselves, ironically). So we'll need to find other trick play assets. Media logs (formatted via this script):

Timestamp	Property	Value
00:00:00 00	pipeline_state	kCreated
00:00:00 00	event	WEBMEDIAPLAYER_CREATED
00:00:00 00	url	blob:http://localhost/6dd4de73-abdd-4c59-ae4f-1d4ec253bb26
00:00:00 00	pipeline_state	kInitDemuxer
00:00:00 229	duration	654
00:00:00 425	info	Audio codec: mp4a.40.2. Sampling frequency: 24000Hz. Sampling frequency(Extension): 48000Hz. Channel layout: 3.
00:00:00 425	found_audio_stream	true
00:00:00 425	audio_codec_name	aac
00:00:00 480	info	Video codec: avc1.4D401F
00:00:00 480	found_video_stream	true
00:00:00 480	video_codec_name	h264
00:00:00 480	pipeline_state	kInitRenderer
00:00:00 482	audio_dds	false
00:00:00 482	audio_decoder	FFmpegAudioDecoder
00:00:00 483	video_dds	false
00:00:00 483	video_decoder	FFmpegVideoDecoder
00:00:00 483	pipeline_state	kPlaying
00:00:00 694	error	video frame with PTS 0us has negative DTS -83333us after applying timestampOffset, handling any discontinuity, and filtering against append window
00:00:00 694	error	Append: stream parsing failed. Data size=131072 append_window_start=0 append_window_end=654.04
00:00:00 698	pipeline_error	chunk demuxer: append failed
00:00:00 698	pipeline_state	kStopping
00:00:00 699	pipeline_state	kStopped

[joeyparrish]

Since the DASH-IF's test assets for trick play do not decode in Chrome, I will need to make a new one. I analyzed the trick play track from the manifest linked above, and I see only I-frames at 1 fps. The segment size is 2s, just as in the normal tracks. This will not be difficult to replicate.

[joeyparrish]

@waqarz added new trick mode vectors here: http://testassets.dashif.org/#testvector/list

Thanks!

[joeyparrish]

This is deferred until after some HLS-related refactoring is done. HLS work is higher priority and may conflict with this, since both affect the internal manifest representation. HLS work is tracked in #279.

[joeyparrish]

Refactoring for HLS is done, so we can now continue work on trick mode.