Dynamic HDR SEI messages in h265 bitstreams do not reach media Codec when played from .ts

[dbPhilips]

[REQUIRED] Issue description

In the 2.11.1 version and all tested previous versions, Dynamic HDR SEI messages in an h265 bitstream (such as defined in e.g. ETSI TS 103 433)
do not all reach the MediaCodec properly when the bitstream is packed in a .ts container. This poses a problem for streams that intend to
indicate video decoder meta data through SEI such as dynamic HDR systems.

It seems related to the current H265Reader.java implementation.

Since all of these dynamic HDR streams I have encountered do use AUD's, the simplest modification seems be to use these when available in a stream
in the sample reader. Will open a pull request to share this modification.

It is also observed that without the AUD modification some of the .ts content does not play smoothly. On Qualcomm SD855-based devices this also
triggers mediacodec errors logged in the logcat at the time of the hiccup.

"03-16 16:37:22.195 947 21162 E OMX-VDEC-1080P: Unable to convey fps info to driver, performance might be affected"

Unfortunately the content with the the hiccup is not content that we own or are allowed to distribute.

[REQUIRED] Reproduction steps

Play Hevc Dynamic HDR .ts video with exoplayer

[REQUIRED] Link to test content

The content was mailed to dev.exoplayer@gmail.com

[REQUIRED] A full bug report captured from the device

No applicable for the given content

[REQUIRED] Version of ExoPlayer being used

2.11.1 ,2.10.5 and 2.10.2

[REQUIRED] Device(s) and version(s) of Android being used

Samsung Tab S6 (Android 9)

[marcbaechinger]

Thanks. I assign to Santiago who is looking after the pull request also.

[ojw28]

@dbPhilips - Please could you re-send the content. It's expired.

[dbPhilips]

Done!

New one is valid up to 25th of April. This the maximum our IT department unfortunately allows.

[ojw28]

Thanks!

The proposed fix in #7115 didn't seem ideal to me. It relied on the presence of AUDs to avoid dropping the prefix SEI NAL units, however AUDs are optional and it should be possible to do the right thing even if they're omitted.

I've merged the two changes ref'd above as an alternative solution. These changes should be able to handle prefix SEI NAL units when AUDs are omitted as well. It also fixes how suffix SEI NAL units are handled, to make sure that they're output in the correct sample as well. Looking at the NAL unit types output in each sample for the test content, it appears the changes do the right thing. Before the pattern looked like this, where each line shows the NAL unit types in a single sample:

NAL types: 32, 33, 34, 39, 39, 39, 39, 21
NAL types: 9
NAL types: 9
NAL types: 1
NAL types: 1
NAL types: 32, 33, 34, 39, 39, 39, 39, 21
NAL types: 9
NAL types: 9
NAL types: 1
NAL types: 1
...

After it looks like this. Note we now include the AUD (35) at the start of each sample, and the prefix SEI NAL units (39). The latter would be included even if the AUDs are omitted in the stream:

NAL types: 35, 32, 33, 34, 39, 39, 39, 39, 21
NAL types: 35, 39, 39, 9
NAL types: 35, 39, 39, 9
NAL types: 35, 39, 39, 1
NAL types: 35, 39, 39, 1
NAL types: 35, 32, 33, 34, 39, 39, 39, 39, 21
NAL types: 35, 39, 39, 9
NAL types: 35, 39, 39, 9
NAL types: 35, 39, 39, 1
NAL types: 35, 39, 39, 1
...

I wasn't able to test user observable behaviour, so please do check out dev-v2 and confirm on your side that the issue really is fixed. Thanks!