Opened 11 years ago
Closed 10 years ago
#3553 closed defect (fixed)
Fix Non-monotonous DTS for flv live stream output
Reported by: | FishB8 | Owned by: | |
---|---|---|---|
Priority: | important | Component: | undetermined |
Version: | git-master | Keywords: | dts flv h264 regression |
Cc: | Blocked By: | ||
Blocking: | Reproduced by developer: | yes | |
Analyzed by developer: | no |
Description
I'm running into problems when using ffmpeg to pull a live rtmp stream, re-encode it, and push the result back out to another location. The problem is that the dts can jump when the source stream is switched within the server that is providing the source for ffmpeg. I would assume that +genpts and +igndts flags should fix the problem, but they do not.
When the streams get switched internally in the source rtmp server, I get a bunch of:
[flv @ 0xdeadbeef] Non-monotonous DTS in output stream [blah blah blah ...]
% ffmpeg -i rtmp://0.0.0.0/foo/bar -fflags +genpts+igndts [...] -f flv -rtmp_live live rtmp://0.0.0.0/hash/bang
ffmpeg needs a means to deal with large jumps in dts gracefully. Currently it totally screws up the output stream. Ideally the origional dts would be tossed and new one generated when re-encoding based off the frame rate / frame base. Even better would be a bitstream filter to rewrite dts that could be used even in cases where you are doing a stream copy instead of re-encoding.
ffmpeg version 2.2.1 (using internal RTMP implementation)
Change History (11)
comment:1 by , 11 years ago
Component: | ffmpeg → undetermined |
---|---|
Keywords: | live removed |
comment:2 by , 11 years ago
Note: to avoid confusion, from here on I'll use "network input/output" in reference to rtmp IO, while "stream" refers to video/audio/data element streams within the format.
I built latest git, no difference
The problem is reproducible with both file and network output. Basically the audio continues, but the video is stopped waiting for a frame with the correct DTS value.
Same thing happens when the source is a file with DTS discontinuity rather than a network input.
Once caveat is that if using stream copy for the codecs rather than re-encoding, the video output seems to continue once the DTS discontinuity occurs, but at a fraction of the proper frame rate. (Audio continues normally)
Here's the commandline / console output:
$ ffmpeg -fflags +genpts -i rtmp://127.0.0.1/switcher/output -c:a libfdk_aac -vbr 3 -ac 2 -ar 48000 -c:v libx264 -g 31 -crf 22 -profile:v high -preset:v medium -trellis 2 -f flv -rtmp_live live rtmp://127.0.0.1/publish/program ffmpeg version N-62374-ge89f3d0 Copyright (c) 2000-2014 the FFmpeg developers built on Apr 12 2014 11:08:18 with gcc 4.8.2 (Gentoo 4.8.2 p1.3r1, pie-0.5.8r1) configuration: --prefix=/usr --libdir=/usr/lib64 --shlibdir=/usr/lib64 --mandir=/usr/share/man --enable-shared --cc=x86_64-pc-linux-gnu-gcc --cxx=x86_64-pc-linux-gnu-g++ --ar=x86_64-pc-linux-gnu-ar --optflags='-O2 -march=native -mfpmath=sse -ftree-vectorize -fomit-frame-pointer -pipe -fprefetch-loop-arrays -floop-block -floop-strip-mine -floop-interchange' --extra-cflags='-O2 -march=native -mfpmath=sse -ftree-vectorize -fomit-frame-pointer -pipe -fprefetch-loop-arrays -floop-block -floop-strip-mine -floop-interchange' --extra-cxxflags='-O2 -march=native -mfpmath=sse -ftree-vectorize -fomit-frame-pointer -pipe -fprefetch-loop-arrays -floop-block -floop-strip-mine -floop-interchange -fvisibility-inlines-hidden -DNDEBUG' --disable-static --enable-gpl --enable-postproc --enable-avfilter --enable-avresample --disable-stripping --enable-nonfree --enable-version3 --enable-nonfree --disable-indev=v4l2 --disable-outdev=v4l2 --disable-indev=oss --disable-outdev=oss --enable-version3 --enable-nonfree --enable-bzlib --disable-runtime-cpudetect --disable-debug --disable-doc --enable-gnutls --enable-hardcoded-tables --enable-iconv --enable-network --enable-openssl --enable-ffplay --enable-vaapi --enable-vdpau --enable-xlib --enable-zlib --enable-libvo-aacenc --enable-libvo-amrwbenc --enable-libmp3lame --enable-libaacplus --enable-libfaac --enable-libtheora --enable-libtwolame --disable-libwavpack --enable-libwebp --enable-libx264 --disable-libx265 --enable-libxvid --enable-libcdio --enable-libiec61883 --enable-libdc1394 --disable-libcaca --disable-openal --enable-opengl --disable-libv4l2 --enable-libpulse --enable-x11grab --disable-libflite --enable-frei0r --enable-fontconfig --enable-ladspa --disable-libass --enable-libfreetype --disable-libsoxr --enable-pthreads --enable-libopencore-amrwb --enable-libopencore-amrnb --enable-libfdk-aac --enable-libopenjpeg --enable-libbluray --disable-libcelt --disable-libgme --enable-libgsm --enable-libmodplug --enable-libopus --disable-libquvi --disable-librtmp --disable-libssh --enable-libschroedinger --enable-libspeex --enable-libvorbis --enable-libvpx --disable-libzvbi --disable-armv5te --disable-armv6 --disable-armv6t2 --disable-neon --disable-vfp --disable-mips32r2 --disable-mipsdspr1 --disable-mipsdspr2 --disable-mipsfpu --disable-altivec --disable-avx --disable-avx2 --disable-fma3 --disable-fma4 --disable-sse42 --cpu=host libavutil 52. 75.100 / 52. 75.100 libavcodec 55. 58.103 / 55. 58.103 libavformat 55. 37.100 / 55. 37.100 libavdevice 55. 13.100 / 55. 13.100 libavfilter 4. 4.100 / 4. 4.100 libavresample 1. 2. 0 / 1. 2. 0 libswscale 2. 6.100 / 2. 6.100 libswresample 0. 18.100 / 0. 18.100 libpostproc 52. 3.100 / 52. 3.100 [flv @ 0xd06450] Stream discovered after head already parsed Input #0, flv, from 'rtmp://127.0.0.1/switcher/output': Metadata: Server : NGINX RTMP (github.com/arut/nginx-rtmp-module) displayWidth : 1280 displayHeight : 720 fps : 29 profile : level : Duration: 00:00:00.00, start: 837.824000, bitrate: N/A Stream #0:0: Video: h264 (High), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], 30.33 fps, 29.97 tbr, 1k tbn, 59.94 tbc Stream #0:1: Audio: aac, 48000 Hz, stereo, fltp Stream #0:2: Data: none [libx264 @ 0xe99550] using SAR=1/1 [libx264 @ 0xe99550] using cpu capabilities: MMX2 SSE2Fast LZCNT [libx264 @ 0xe99550] profile High, level 3.1 [libx264 @ 0xe99550] 264 - core 142 - H.264/MPEG-4 AVC codec - Copyleft 2003-2014 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=2 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=9 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=31 keyint_min=3 scenecut=40 intra_refresh=0 rc_lookahead=31 rc=crf mbtree=1 crf=22.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00 [libfdk_aac @ 0xe95130] Note, the VBR setting is unsupported and only works with some parameter combinations Output #0, flv, to 'rtmp://127.0.0.1/publish/program': Metadata: Server : NGINX RTMP (github.com/arut/nginx-rtmp-module) displayWidth : 1280 displayHeight : 720 fps : 29 profile : level : encoder : Lavf55.37.100 Stream #0:0: Video: h264 (libx264) ([7][0][0][0] / 0x0007), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], q=-1--1, 1k tbn, 29.97 tbc Stream #0:1: Audio: aac (libfdk_aac) ([10][0][0][0] / 0x000A), 48000 Hz, stereo, s16 Stream mapping: Stream #0:0 -> #0:0 (h264 -> libx264) Stream #0:1 -> #0:1 (aac -> libfdk_aac) Press [q] to stop, [?] for help [h264 @ 0xd0f570] AVC: nal size 23330847kB time=00:00:03.92 bitrate= 484.4kbits/s [h264 @ 0xd0f570] missing picture in access unit with size 35 [flv @ 0xd06450] DTS 0 < 841784 out of order [libfdk_aac @ 0xe95130] Queue input is backward in time [flv @ 0x141dd70] Non-monotonous DTS in output stream 0:1; previous: 3947, current: -837781; changing to 3947. This may result in incorrect timestamps in the output file. (Repeats flv DTS warning for each frame)
I can provide example files (both source and output) and an nginx-rtmp config file to re-create this issue if needed.
comment:3 by , 11 years ago
Version: | 2.2 → git-master |
---|
The problem is reproducible with both file and network output.
Same thing happens when the source is a file with DTS discontinuity rather than a network input.
Please understand that issues involving the network are a magnitude more difficult to reproduce than bugs concerning files so please provide an input file sample (and a failing command line for that sample including complete, uncut console output) to allow us to reproduce the problem.
comment:4 by , 11 years ago
Ok, I'm going attach an example file. The following is the result of trying to do the exact same as was shown previously, just using files instead of network rtmp IO.
ffmpeg -fflags +genpts+igndts -i DTS_Discontinuity.flv -c:a libfdk_aac -vbr 3 -ac 2 -ar 48000 -c:v libx264 -g 31 -crf 22 -profile:v high -preset:v medium -trellis 2 -f flv DTS_Test.flv ffmpeg version N-62374-ge89f3d0 Copyright (c) 2000-2014 the FFmpeg developers built on Apr 12 2014 11:08:18 with gcc 4.8.2 (Gentoo 4.8.2 p1.3r1, pie-0.5.8r1) configuration: --prefix=/usr --libdir=/usr/lib64 --shlibdir=/usr/lib64 --mandir=/usr/share/man --enable-shared --cc=x86_64-pc-linux-gnu-gcc --cxx=x86_64-pc-linux-gnu-g++ --ar=x86_64-pc-linux-gnu-ar --optflags='-O2 -march=native -mfpmath=sse -ftree-vectorize -fomit-frame-pointer -pipe -fprefetch-loop-arrays -floop-block -floop-strip-mine -floop-interchange' --extra-cflags='-O2 -march=native -mfpmath=sse -ftree-vectorize -fomit-frame-pointer -pipe -fprefetch-loop-arrays -floop-block -floop-strip-mine -floop-interchange' --extra-cxxflags='-O2 -march=native -mfpmath=sse -ftree-vectorize -fomit-frame-pointer -pipe -fprefetch-loop-arrays -floop-block -floop-strip-mine -floop-interchange -fvisibility-inlines-hidden -DNDEBUG' --disable-static --enable-gpl --enable-postproc --enable-avfilter --enable-avresample --disable-stripping --enable-nonfree --enable-version3 --enable-nonfree --disable-indev=v4l2 --disable-outdev=v4l2 --disable-indev=oss --disable-outdev=oss --enable-version3 --enable-nonfree --enable-bzlib --disable-runtime-cpudetect --disable-debug --disable-doc --enable-gnutls --enable-hardcoded-tables --enable-iconv --enable-network --enable-openssl --enable-ffplay --enable-vaapi --enable-vdpau --enable-xlib --enable-zlib --enable-libvo-aacenc --enable-libvo-amrwbenc --enable-libmp3lame --enable-libaacplus --enable-libfaac --enable-libtheora --enable-libtwolame --disable-libwavpack --enable-libwebp --enable-libx264 --disable-libx265 --enable-libxvid --enable-libcdio --enable-libiec61883 --enable-libdc1394 --disable-libcaca --disable-openal --enable-opengl --disable-libv4l2 --enable-libpulse --enable-x11grab --disable-libflite --enable-frei0r --enable-fontconfig --enable-ladspa --disable-libass --enable-libfreetype --disable-libsoxr --enable-pthreads --enable-libopencore-amrwb --enable-libopencore-amrnb --enable-libfdk-aac --enable-libopenjpeg --enable-libbluray --disable-libcelt --disable-libgme --enable-libgsm --enable-libmodplug --enable-libopus --disable-libquvi --disable-librtmp --disable-libssh --enable-libschroedinger --enable-libspeex --enable-libvorbis --enable-libvpx --disable-libzvbi --disable-armv5te --disable-armv6 --disable-armv6t2 --disable-neon --disable-vfp --disable-mips32r2 --disable-mipsdspr1 --disable-mipsdspr2 --disable-mipsfpu --disable-altivec --disable-avx --disable-avx2 --disable-fma3 --disable-fma4 --disable-sse42 --cpu=host libavutil 52. 75.100 / 52. 75.100 libavcodec 55. 58.103 / 55. 58.103 libavformat 55. 37.100 / 55. 37.100 libavdevice 55. 13.100 / 55. 13.100 libavfilter 4. 4.100 / 4. 4.100 libavresample 1. 2. 0 / 1. 2. 0 libswscale 2. 6.100 / 2. 6.100 libswresample 0. 18.100 / 0. 18.100 libpostproc 52. 3.100 / 52. 3.100 Input #0, flv, from 'DTS_Discontinuity.flv': Metadata: Server : NGINX RTMP (github.com/arut/nginx-rtmp-module) displayWidth : 1280 displayHeight : 720 fps : 29 profile : level : Duration: 00:00:06.28, start: 8.854000, bitrate: 4266 kb/s Stream #0:0: Video: h264 (High), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], 30.33 fps, 29.97 tbr, 1k tbn, 59.94 tbc Stream #0:1: Audio: aac, 48000 Hz, stereo, fltp File 'DTS_Test.flv' already exists. Overwrite ? [y/N] y [libx264 @ 0x1c5b1f0] using SAR=1/1 [libx264 @ 0x1c5b1f0] using cpu capabilities: MMX2 SSE2Fast LZCNT [libx264 @ 0x1c5b1f0] profile High, level 3.1 [libx264 @ 0x1c5b1f0] 264 - core 142 - H.264/MPEG-4 AVC codec - Copyleft 2003-2014 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=2 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=9 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=31 keyint_min=3 scenecut=40 intra_refresh=0 rc_lookahead=31 rc=crf mbtree=1 crf=22.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00 [libfdk_aac @ 0x1c5c200] Note, the VBR setting is unsupported and only works with some parameter combinations Output #0, flv, to 'DTS_Test.flv': Metadata: Server : NGINX RTMP (github.com/arut/nginx-rtmp-module) displayWidth : 1280 displayHeight : 720 fps : 29 profile : level : encoder : Lavf55.37.100 Stream #0:0: Video: h264 (libx264) ([7][0][0][0] / 0x0007), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], q=-1--1, 1k tbn, 29.97 tbc Stream #0:1: Audio: aac (libfdk_aac) ([10][0][0][0] / 0x000A), 48000 Hz, stereo, s16 Stream mapping: Stream #0:0 -> #0:0 (h264 -> libx264) Stream #0:1 -> #0:1 (aac -> libfdk_aac) Press [q] to stop, [?] for help [h264 @ 0x15697a0] AVC: nal size 23330847B time=00:00:04.69 bitrate= 247.0kbits/s [h264 @ 0x15697a0] missing picture in access unit with size 35 [flv @ 0x15613d0] DTS 0 < 15225 out of order [libfdk_aac @ 0x1c5c200] Queue input is backward in time [flv @ 0x1c5a960] Non-monotonous DTS in output stream 0:1; previous: 6357, current: -8811; changing to 6357. This may result in incorrect timestamps in the output file. [...] [flv @ 0x1c5a960] Non-monotonous DTS in output stream 0:1; previous: 6357, current: -2582; changing to 6357. This may result in incorrect timestamps in the output file. frame= 191 fps= 99 q=-1.0 Lsize= 323kB time=00:00:06.37 bitrate= 415.1kbits/s dup=0 drop=188 video:271kB audio:38kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 4.572499% [libx264 @ 0x1c5b1f0] frame I:7 Avg QP:12.04 size: 7908 [libx264 @ 0x1c5b1f0] frame P:184 Avg QP:16.58 size: 1205 [libx264 @ 0x1c5b1f0] mb I I16..4: 70.8% 26.9% 2.3% [libx264 @ 0x1c5b1f0] mb P I16..4: 5.9% 0.6% 0.0% P16..4: 7.4% 0.4% 0.1% 0.0% 0.0% skip:85.5% [libx264 @ 0x1c5b1f0] 8x8 transform intra:15.9% inter:70.5% [libx264 @ 0x1c5b1f0] coded y,uvDC,uvAC intra: 1.9% 25.7% 6.1% inter: 0.3% 5.2% 0.4% [libx264 @ 0x1c5b1f0] i16 v,h,dc,p: 83% 10% 1% 6% [libx264 @ 0x1c5b1f0] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 47% 15% 33% 1% 0% 1% 0% 2% 0% [libx264 @ 0x1c5b1f0] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 27% 36% 29% 2% 1% 2% 0% 2% 1% [libx264 @ 0x1c5b1f0] i8c dc,h,v,p: 30% 11% 49% 10% [libx264 @ 0x1c5b1f0] Weighted P-Frames: Y:0.0% UV:0.0% [libx264 @ 0x1c5b1f0] ref P L0: 58.9% 2.5% 31.3% 7.3% [libx264 @ 0x1c5b1f0] kb/s:347.72
comment:5 by , 11 years ago
The file was slightly too big, so I pushed it to the upload.ffmpeg.org ftp site.
File's name is "BUG-3553_DTS_Discontinuity.flv"
comment:6 by , 11 years ago
Keywords: | flv h264 regression added; rtmp removed |
---|---|
Priority: | normal → important |
Reproduced by developer: | set |
Status: | new → open |
Regression since c5ea3a00 - related to ticket #1627,
Reproducible with the following command lines:
$ ffmpeg -i BUG-3553_DTS_Discontinuity.flv -qscale 2 -strict -2 -acodec aac out.flv $ ffmpeg -i BUG-3553_DTS_Discontinuity.flv -qscale 2 -strict -2 out.mov $ ffmpeg -i BUG-3553_DTS_Discontinuity.flv -qscale 2 out.avi
comment:7 by , 11 years ago
does this issue also happen with librtmp instead of the inernal rtmp code ?
does it happen with other rtmp servers than nginx rtmp ?
ive never seen such timestamp resets in flv files before so iam not sure this file is valid
Do you know of some document that describes how such timestamp discontinuities should be handled in rtmp ?
also to quote the flv spec:
Timestamp UI24 Time in milliseconds at which the data in this tag applies. This value is relative to the first tag in the FLV file, which always has a timestamp of 0.
This does not sound to me like it ever may go backward
comment:8 by , 11 years ago
Yes, this happens with both. I actually started with librtmp, and then switched to internal to see if it made any difference. (Off-topic: which of those 2 is recommended? which gets more dev attention?)
I have not tried it with other rtmp servers other than nginx. The timestamp reset happens when the rtmp server internally switches between source inputs. (In the example file, both sources were live) Here is a link to the blog post when the feature was introduced: http://nginx-rtmp.blogspot.com/2014/01/redirecting-streams-in-version-111.html
I'm not very knowledgeable of what is and isn't allowed in the rtmp spec, so if the nginx implementation is a departure from the spec, let me know and I will submit a bug report with that project. I'll ask the developer what documentation he's basing the feature off of. There is always the chance that I am not utilizing the feature correctly.
comment:10 by , 11 years ago
I've cross-filed a bug report with the nginx rtmp server tracker. https://github.com/arut/nginx-rtmp-module/issues/385 Hopefully Arutyunyan can provide more insight to this than me.
comment:11 by , 10 years ago
Resolution: | → fixed |
---|---|
Status: | open → closed |
Workaround added/fixed in 2fbdfba0f2d1851f894002d9d5930799629cc194
Is the problem also reproducible with file input instead of network input?
Is the problem also reproducible with file output instead of network output?
Please test current git head and please provide your failing command line together with the complete, uncut console output to make this a valid ticket.