HEVC: 1920x1080i file decoded as 1920x540p

[Underground78]

Summary of the bug:
It seems ffmpeg is decoding ​this sample as 1920x540 progressive instead of 1920x1080 interlaced.

How to reproduce:

ffmpeg version N-67999-gd0879a9 Copyright (c) 2000-2014 the FFmpeg developers
  built on Nov 25 2014 11:37:18 with gcc 4.9.2 (GCC)
  configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libschroedinger --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-aacenc --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-decklink --enable-zlib
  libavutil      54. 15.100 / 54. 15.100
  libavcodec     56. 13.100 / 56. 13.100
  libavformat    56. 15.100 / 56. 15.100
  libavdevice    56.  3.100 / 56.  3.100
  libavfilter     5.  2.103 /  5.  2.103
  libswscale      3.  1.101 /  3.  1.101
  libswresample   1.  1.100 /  1.  1.100
  libpostproc    53.  3.100 / 53.  3.100
[hevc @ 02a52aa0] Could not find ref with POC 72
[hevc @ 02a52aa0] Could not find ref with POC 68
[hevc @ 02a52aa0] Could not find ref with POC 64
Input #0, mpeg, from 'Service 5 (T2 Test) (2014-11-26 01-39-13-695).mpg':
  Duration: 00:00:04.69, start: 50223.448456, bitrate: 3204 kb/s
    Stream #0:0[0x1e0]: Video: hevc (Main), yuv420p(tv), 1920x540, 25 fps, 25 tbr, 90k tbn, 25 tbc
    Stream #0:1[0x80]: Audio: eac3, 48000 Hz, stereo, fltp, 192 kb/s

[robert_s]

Is it possible that the H.265/HEVC decoder in ffmpeg cannot handle interlaced content yet? I do not see the progressive_source_flag, interlaced_source_flag or frame_only_constraint flag being evaluated anywhere in the source code...

The "pic_height_in_luma_samples" being 544 seems to be right, as according to page 8 of ITU-T H.265/AVC:

"NOTE – A picture may be either a frame or a field."

So a 1080i stream, encoded in H.265/HEVC as 1088i, a field = picture would indeed have a height of 544 luma samples.

[Hendrik]

The progressive_source_flag or interlaced_source_flag are only hints on the origin of the material, not an actual indicator that the video is encoded as interlaced fields.

The only field of interest is the field_seq_flag in the VUI, and the picture timing SEIs, which are not being used to combine fields back into frame as it is (but they are parsed).

[robert_s]

Thanks for the pointer to the flag that matters, I was a bit confused which of the flags it was.

So the assumption was correct that support for interlaced HEVC streams is not (fully) implemented yet and the observed behavior is to be expected from the current implementation?

[Carl Eugen Hoyos]

Replying to robert_s:

So the assumption was correct that support for interlaced HEVC streams is not (fully) implemented yet and the observed behavior is to be expected from the current implementation?

The reference decoder does the same as FFmpeg.

[robert_s]

Taking a closer look at the sample stream, it appears that field_seq_flag is 1 in the VUI, but the SEI indicates pic_struct = 3, for which Table D.2 of H.265 says "field_seq_flag shall be 0"...

What does "pic_struct = 3" actually mean? That one picture is both top and bottom field, i.e. that the "deinterlaced" output would have every line of the picture doubled?

How would that be implemented in ffmpeg? Can that output method be specified in AVFrame, or does the decoder need to do this before passing the AVFrame?

I found that H.264 has the same value for pic_struct, so I looked at the H.264 decoder sources how pic_struct = 3 is handled there, but it doesn't seem to do anything other than indicate interlaced content. Am I missing something or does the H.264 decoder also not support this pic_struct value properly yet?

[Hendrik]

if field_seq_flag is 1 and pic_struct is 3, then the encode is invalid, as it does not conform to the specification which doesn't allow this combination.

As I understand it, 3 and 4 are meant for cases where both fields are in one frame (weaved), and you don't need to do more than to indicate interlaced and field order in ffmpeg, as thats what the API expects anyway.

[robert_s]

Replying to heleppkes:

if field_seq_flag is 1 and pic_struct is 3, then the encode is invalid, as it does not conform to the specification which doesn't allow this combination.

I wonder how well that would actually work. Without field_seq_flag indicated in the SPS, the decoder would not know the output size of the video until it gets a SEI message...

Replying to heleppkes:

As I understand it, 3 and 4 are meant for cases where both fields are in one frame (weaved), and you don't need to do more than to indicate interlaced and field order in ffmpeg, as thats what the API expects anyway.

So I assume the decoder needs to do the following if pic_struct == 3:

1. Decode picture
2. Stretch decoded picture vertically by doubling each line
3. Set frame->top_field_first = 1 (would be 0 for pic_struct == 4)
4. Set frame->interlaced_frame = 1

Is that the way, or is there some way the decoder can omit step 2 and tell ffmpeg to do the line doubling instead...?

[Carl Eugen Hoyos]

I remuxed the sample with FFmpeg to mpegts and a Samsung hardware decoder plays the sample in full resolution, so I believe this should be fixed.

[cgbug]

Another sample:
​http://www.datafilehost.com/d/db5e55a2

[jkqxz]

Replying to cgbug:

Another sample:
​http://www.datafilehost.com/d/db5e55a2

cgbug: Would it be acceptable for this sample (or a trimmed version of it) to be included in the FFmpeg FATE test suite?

[cgbug]

Yes, go ahead.

[Curtisb]

ATSC 3.0 service has started to launch in a handful of markets and unfortunately some of the stations are still sending interlaced content. This is a needed enhancement/correction of FFMpeg because it currently makes any of the players that use FFmpeg to be unusable for these channels (Such as VLC and Plex).
Here is a clip (49 seconds, 41 MB) from CBS captured with VLC which is broadcasting HVEC 1080i:

​https://1drv.ms/f/s!Ap22nCAxKBS7j-Jr9Q8eX45gp2Uh7w

[Balling]

Duplicate of #5514.

This bug

this is not a bug. Interlaced hevc is very much a thing. Even Mediainfo does not recognize such files.

[Aman]

The link to the mediainfo bug is actually ​https://github.com/MediaArea/MediaInfoLib/issues/1092

[Curtisb]

Replying to Balling:

Duplicate of #5514.

This bug

this is not a bug. Interlaced hevc is very much a thing. Even Mediainfo does not recognize such files. There is a link to mediainfo issue about that in #5514.

The "bug" I was referring to is this ticket #4141, not calling interlaced HEVC a bug (though I wish they didn't use it). I'll change it to 'Needed enhancement of FFMpeg" to clarify.

[Curtisb]

Replying to Balling:

Duplicate of #5514.

This bug

this is not a bug. Interlaced hevc is very much a thing. Even Mediainfo does not recognize such files. There is a link to mediainfo issue about that in #5514.

The "bug" I was refering to is this ticket #4141, not calling interlaced HEVC a bug (though I wish they didn't use it). I'll change it to 'Needed enhancement of FFMpeg" to clarify.

[kasper93]

Another sample from DVB-T2 HEVC broadcast ​https://www.mediafire.com/file/dm849hm4ykqyzvj Don't know how widespread are interlaced HEVC broadcasts, sending sample for reference as original one is not available.

[Balling]

Replying to kasper93:

Another sample from DVB-T2 HEVC broadcast ​https://www.mediafire.com/file/dm849hm4ykqyzvj Don't know how widespread are interlaced HEVC broadcasts, sending sample for reference as original one is not available.

​https://samples.ffmpeg.org/ffmpeg-bugs/trac/ticket4141/