Opened 10 years ago
Closed 10 years ago
#3724 closed defect (fixed)
Computed HLS duration is sometimes incorrect with short segment durations
Reported by: | fthiery | Owned by: | |
---|---|---|---|
Priority: | normal | Component: | avformat |
Version: | git-master | Keywords: | hls segment |
Cc: | Blocked By: | ||
Blocking: | Reproduced by developer: | no | |
Analyzed by developer: | no |
Description
Summary of the bug:
When segmenting an mp4/H.264/AAC file, computed HLS playlist duration (by summing all EXTINF metadata) deviates from the proper duration, i.e. the original file's.
How to reproduce:
% ffmpeg -i video.mp4 -flags -global_header -f segment -segment_time 1 -segment_list playlist.m3u8 -segment_list_type m3u8 -segment_format mpegts -map 0 -bsf h264_mp4toannexb -vcodec copy -acodec copy -y media%05d.ts Ffmpeg version compiled from git today: ffmpeg version N-64012-g61df081 Copyright (c) 2000-2014 the FFmpeg developers built on Jun 17 2014 08:50:14 with gcc 4.8 (Ubuntu 4.8.1-2ubuntu1~10.04.1) configuration: --disable-ffserver --enable-shared --enable-libx264 --enable-libfaac --enable-libtheora --enable-libvorbis --enable-libmp3lame --enable-libxvid --enable-gpl --enable-nonfree --enable-pthreads --enable-libvpx --extra-cflags='-I/tmp/codecs/include/ -I/tmp/codecs/include/ao -I/tmp/codecs/include/lame -I/tmp/codecs/include/ogg -I/tmp/codecs/include/vorbis -I/tmp/codecs/include/theora -I/tmp/codecs/include/vpx' --extra-ldflags='-L/tmp/codecs/lib -static' libavutil 52. 89.100 / 52. 89.100 libavcodec 55. 67.100 / 55. 67.100 libavformat 55. 43.100 / 55. 43.100 libavdevice 55. 13.101 / 55. 13.101 libavfilter 4. 8.100 / 4. 8.100 libswscale 2. 6.100 / 2. 6.100 libswresample 0. 19.100 / 0. 19.100 libpostproc 52. 3.100 / 52. 3.100
ffprobe reports some duration on the m3u8 resource (in my case, 1957.493063s) where the actual duration is 1935.033333s (obtained by ffprob'ing the original mp4 file or using the last frame timestamp + frame duration by analyzing the m3u8 resource with ffprobe). One of the issues is that the duration difference is large enough (~40s for one hour of content) for most players to seek to an unappropriate location (e.g. 10 seconds too late).
Btw original mp4 file has an average gop size of 30 (video is 30 fps), and the goal here is to have 1 second segments for precise seeking.
Change History (11)
comment:1 by , 10 years ago
Summary: | Computed HLS duration is incorrect (deviates by ~700ms per minute) → Computed HLS duration is incorrect with short segment durations |
---|
comment:2 by , 10 years ago
Is a specific input file necessary to reproduce the issue or does ffmpeg -f lavfi -i testsrc
allow to reproduce?
Please provide the command line that produces the unexpected output files together with the complete, uncut console output to make this a valid ticket.
follow-up: 4 comment:3 by , 10 years ago
I am indeed failing to reproduce it using a clean, generated test file.
ffmpeg -f lavfi -i testsrc -t 1800 -r 30 -g 30 -codec:v libx264 -pix_fmt yuv420p -b:v 1M -filter_complex "aevalsrc=sin(440*2*PI*t)" -t 1800 -c:a libfaac -b:a 128k test.mp4
Duration is quite precisely 1800s (30 min):
ffprobe test.mp4 [...] Duration: 00:30:00.02, start: 0.023220, bitrate: 313 kb/s
Then i segment:
mkdir ts ffmpeg -i test.mp4 -flags -global_header -f segment -segment_time 1 -segment_list ts/playlist.m3u8 -segment_list_type m3u8 -segment_format mpegts -map 0 -bsf h264_mp4toannexb -vcodec copy -acodec copy -y ts/media%05d.ts
Duration is then correct
ffprobe ts/playlist.m3u [...] Duration: 00:30:00.09, start: 0.043422, bitrate: 0 kb/s
The difference is that my content is the concatenation of two mkv files using mkvtoolnix's mkvmerge between a 5 seconds long "intro" and the actual video. Analyzing the gopsize using ffprobe shows that gops get "offset" after the intro video, where the file generated with ffmpeg has a perfect gop of 30 all the time.
Gopsize (i.e. timecode_s is the iframe timecode, gopsize is the number of consecutive frames + 1 until the next keyframe):
timecode_s | gopsize |
0.0 | 30 |
1.0 | 30 |
2.0 | 30 |
3.0 | 30 |
4.0 | 30 |
5.0 | 7 |
5.233 | 30 |
6.233 | 30 |
... | ... |
94.233 | 30 |
95.233 | 2 |
95.3 | 28 |
96.233 | 30 |
... | ... |
220.833 | 30 |
221.833 | 1 |
221.866 | 29 |
686.833 | 30 |
687.833 | 1 |
687.866 | 29 |
688.833 | 30 |
... | ... |
While i am suspecting that this merging is the origin of the problem, is there a different way of generating a test video using ffmpeg so that it triggers scene changes and shorter gops (as gops of 1 or 2, then 28 in the real video) ?
follow-up: 7 comment:4 by , 10 years ago
Replying to fthiery:
The difference is that my content is the concatenation of two mkv files using mkvtoolnix's mkvmerge
How did you convert the output of mkvmerge to the mp4 file used in your original report?
comment:5 by , 10 years ago
After further investigation, it seems that the merging is not related, but the encoding.
After remuxing, reencoding audio, then video, it seems that the original h.264 encoding is
the problem (gstreamer's x264enc with pass=5 quantizer=22 subme=4 key-int-max=30) because the problem only appears if it keep the original video bitstream (vcodec copy).
The closest equivalent would be -r 30 -codec:v libx264 -crf 22 -x264opts keyint=30:subme=4, and it still does not reproduce the issue...
On the reencoded file, there are non-regular gops (on scenecuts), but the problem doesn't appear.
89.0 | 30 |
90.0 | 2 |
90.066667 | 28 |
91.0 | 30 |
92.0 | 30 |
If you are willing to take a look, the following video triggers the problem: http://d2hqzi4c71e9y2.cloudfront.net/moodlemoot/presentation-des-defis_83f8/presentation-des-defis_low.mp4
comment:6 by , 10 years ago
Please post the command line that produces the unexpected output for the sample you provided together with the complete, uncut console output.
comment:7 by , 10 years ago
Here it is:
wget http://d2hqzi4c71e9y2.cloudfront.net/moodlemoot/presentation-des-defis_83f8/presentation-des-defis_low.mp4 ffprobe presentation-des-defis_low.mp4 ffprobe version N-60597-g1e5cb42 Copyright (c) 2007-2014 the FFmpeg developers built on Feb 14 2014 10:22:18 with gcc 4.8 (Ubuntu 4.8.1-2ubuntu1~10.04.1) configuration: --disable-ffserver --enable-shared --enable-libx264 --enable-libfaac --enable-libtheora --enable-libvorbis --enable-libmp3lame --enable-libxvid --enable-gpl --enable-nonfree --enable-pthreads --enable-libvpx --extra-cflags='-I/tmp/codecs/include/ -I/tmp/codecs/include/ao -I/tmp/codecs/include/lame -I/tmp/codecs/include/ogg -I/tmp/codecs/include/vorbis -I/tmp/codecs/include/theora -I/tmp/codecs/include/vpx' --extra-ldflags='-L/tmp/codecs/lib -static' libavutil 52. 63.101 / 52. 63.101 libavcodec 55. 52.101 / 55. 52.101 libavformat 55. 32.101 / 55. 32.101 libavdevice 55. 9.101 / 55. 9.101 libavfilter 4. 1.102 / 4. 1.102 libswscale 2. 5.101 / 2. 5.101 libswresample 0. 17.104 / 0. 17.104 libpostproc 52. 3.100 / 52. 3.100 Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'presentation-des-defis_low.mp4': Metadata: major_brand : mp42 minor_version : 0 compatible_brands: mp42mp41isomiso2 creation_time : 2014-06-17 20:14:44 Duration: 00:05:11.96, start: 0.000000, bitrate: 769 kb/s Stream #0:0(eng): Audio: aac (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 96 kb/s (default) Metadata: creation_time : 2014-06-17 20:14:44 handler_name : SoundHandler Stream #0:1(eng): Video: h264 (Main) (avc1 / 0x31637661), yuv420p, 640x360 [SAR 1:1 DAR 16:9], 669 kb/s, 25 fps, 25 tbr, 1k tbn, 50 tbc (default) Metadata: creation_time : 2014-06-17 20:14:44 handler_name : VideoHandler mkdir ts ffmpeg -i presentation-des-defis_low.mp4 -flags -global_header -f segment -segment_time 1 -segment_list ts/playlist.m3u8 -segment_list_type m3u8 -segment_format mpegts -map 0 -bsf h264_mp4toannexb -vcodec copy -acodec copy -y ts/media%05d.ts ffmpeg version N-64012-g61df081 Copyright (c) 2000-2014 the FFmpeg developers built on Jun 17 2014 08:50:14 with gcc 4.8 (Ubuntu 4.8.1-2ubuntu1~10.04.1) configuration: --disable-ffserver --enable-shared --enable-libx264 --enable-libfaac --enable-libtheora --enable-libvorbis --enable-libmp3lame --enable-libxvid --enable-gpl --enable-nonfree --enable-pthreads --enable-libvpx --extra-cflags='-I/tmp/codecs/include/ -I/tmp/codecs/include/ao -I/tmp/codecs/include/lame -I/tmp/codecs/include/ogg -I/tmp/codecs/include/vorbis -I/tmp/codecs/include/theora -I/tmp/codecs/include/vpx' --extra-ldflags='-L/tmp/codecs/lib -static' libavutil 52. 89.100 / 52. 89.100 libavcodec 55. 67.100 / 55. 67.100 libavformat 55. 43.100 / 55. 43.100 libavdevice 55. 13.101 / 55. 13.101 libavfilter 4. 8.100 / 4. 8.100 libswscale 2. 6.100 / 2. 6.100 libswresample 0. 19.100 / 0. 19.100 libpostproc 52. 3.100 / 52. 3.100 Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'presentation-des-defis_low.mp4': Metadata: major_brand : mp42 minor_version : 0 compatible_brands: mp42mp41isomiso2 creation_time : 2014-06-17 20:14:44 Duration: 00:05:11.96, start: 0.000000, bitrate: 769 kb/s Stream #0:0(eng): Audio: aac (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 96 kb/s (default) Metadata: creation_time : 2014-06-17 20:14:44 handler_name : SoundHandler Stream #0:1(eng): Video: h264 (Main) (avc1 / 0x31637661), yuv420p, 640x360 [SAR 1:1 DAR 16:9], 669 kb/s, 25 fps, 25 tbr, 1k tbn, 50 tbc (default) Metadata: creation_time : 2014-06-17 20:14:44 handler_name : VideoHandler [segment @ 0xa0deac0] Codec for stream 0 does not use global headers but container format requires global headers [segment @ 0xa0deac0] Codec for stream 1 does not use global headers but container format requires global headers Output #0, segment, to 'ts/media%05d.ts': Metadata: major_brand : mp42 minor_version : 0 compatible_brands: mp42mp41isomiso2 encoder : Lavf55.43.100 Stream #0:0(eng): Audio: aac (mp4a / 0x6134706D), 44100 Hz, stereo, 96 kb/s (default) Metadata: creation_time : 2014-06-17 20:14:44 handler_name : SoundHandler Stream #0:1(eng): Video: h264 (avc1 / 0x31637661), yuv420p, 640x360 [SAR 1:1 DAR 16:9], q=2-31, 669 kb/s, 25 fps, 90k tbn, 25 tbc (default) Metadata: creation_time : 2014-06-17 20:14:44 handler_name : VideoHandler Stream mapping: Stream #0:0 -> #0:0 (copy) Stream #0:1 -> #0:1 (copy) Press [q] to stop, [?] for help frame= 7799 fps=6601 q=-1.0 Lsize=N/A time=00:05:11.96 bitrate=N/A video:25496kB audio:3655kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown ffprobe ts/playlist.m3u ffprobe version N-60597-g1e5cb42 Copyright (c) 2007-2014 the FFmpeg developers built on Feb 14 2014 10:22:18 with gcc 4.8 (Ubuntu 4.8.1-2ubuntu1~10.04.1) configuration: --disable-ffserver --enable-shared --enable-libx264 --enable-libfaac --enable-libtheora --enable-libvorbis --enable-libmp3lame --enable-libxvid --enable-gpl --enable-nonfree --enable-pthreads --enable-libvpx --extra-cflags='-I/tmp/codecs/include/ -I/tmp/codecs/include/ao -I/tmp/codecs/include/lame -I/tmp/codecs/include/ogg -I/tmp/codecs/include/vorbis -I/tmp/codecs/include/theora -I/tmp/codecs/include/vpx' --extra-ldflags='-L/tmp/codecs/lib -static' libavutil 52. 63.101 / 52. 63.101 libavcodec 55. 52.101 / 55. 52.101 libavformat 55. 32.101 / 55. 32.101 libavdevice 55. 9.101 / 55. 9.101 libavfilter 4. 1.102 / 4. 1.102 libswscale 2. 5.101 / 2. 5.101 libswresample 0. 17.104 / 0. 17.104 libpostproc 52. 3.100 / 52. 3.100 Input #0, hls,applehttp, from 'ts/playlist.m3u8': Duration: 00:05:15.51, start: 0.000000, bitrate: 0 kb/s Program 0 Metadata: variant_bitrate : 0 Stream #0:0: Audio: aac ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp, 137 kb/s Stream #0:1: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p, 640x360 [SAR 1:1 DAR 16:9], 25 fps, 25 tbr, 90k tbn, 50 tbc
As you can see, duration of original video is 00:05:11.96, where the HLS segmented version is 00:05:15.51, a 4 seconds difference (around 13ms per second of added duration)
comment:8 by , 10 years ago
Summary: | Computed HLS duration is incorrect with short segment durations → Computed HLS duration is sometimes incorrect with short segment durations |
---|
comment:10 by , 10 years ago
Replying to fthiery:
[...]
ffprobe reports some duration on the m3u8 resource (in my case, 1957.493063s) where the actual duration is 1935.033333s (obtained by ffprob'ing the original mp4 file or using the last frame timestamp + frame duration by analyzing the m3u8 resource with ffprobe). One of the issues is that the duration difference is large enough (~40s for one hour of content) for most players to seek to an unappropriate location (e.g. 10 seconds too late).
Btw original mp4 file has an average gop size of 30 (video is 30 fps), and the goal here is to have 1 second segments for precise seeking.
I think this is due to the way the segment muxer computes the duration of each segment.
We have this code to compute the ending time (libavformat/segment.c):
seg->cur_entry.end_time = FFMAX(seg->cur_entry.end_time, (double)(pkt->pts + pkt->duration) * av_q2d(st->time_base));
The adopted semantics is: total segment duration = maximum ending time of contained packets minus initial time of the segment. If you have several streams it might be that the sum of the segments durations will be greater to the real global duration (for a single stream or for an A/V stream where A/V durations are aligned this shouldn't be an issue, so you won't be able to reproduce the issue with a synthetic testsrc
stream).
Note that when computing the HLS duration (libavformat/hls.c) is simply performing the sum of the duration of each segment as read from the EXTINF tag.
A possible solution would be to compute the duration only considering the reference stream packets. Not sure if this is consistent with the specs though.
comment:11 by , 10 years ago
Component: | undetermined → avformat |
---|---|
Keywords: | segment added |
Resolution: | → fixed |
Status: | new → closed |
Should be fixed now in commit:
commit 5acad50056f653672a0e9b6b2986089c32046afb Author: Stefano Sabatini <stefasab@gmail.com> Date: Thu Jul 17 14:49:20 2014 +0200 lavf/segment: only use reference frames for computing the segment end time This avoids a systematic overestimate of the segments duration when there are several streams. Fix trac ticket #3724.
Example: real duration is 1935.03333 seconds.
When raising the segment duration, the duration difference (and drift) is smaller.