Opened 6 years ago
Last modified 4 years ago
#7547 new defect
When relaying RTMP streams ffmpeg quits with av_interleaved_write_frame message
Reported by: | regstuff | Owned by: | |
---|---|---|---|
Priority: | important | Component: | avformat |
Version: | git-master | Keywords: | regression librtmp |
Cc: | 9036178@qq.com | Blocked By: | |
Blocking: | Reproduced by developer: | no | |
Analyzed by developer: | no |
Description
Summary of the bug: I have an Nginx server (with rtmp-module) on a Ubuntu-16.04 VM. I'm using it to receive an input rtmp stream from my PC. FFmpeg is then used in the VM to take this input and relay to multiple destinations after transcoding, overlaying a lower third, and changing the volume.
However, FFmpeg quits after a random duration between 3-30 minutes with an "av_interleaved_write_frame(): End of file" message.
How to reproduce: Pull an rtmp stream as input. Command used was as below. In the below command, I have used aac and strict -2 as well. I am aware that strict -2 is no longer necessary. The error occurs even with other aac encoders such as libfdk_aac and without strict -2. The reason I have used this particular command is explained in more detail below.
/usr/local/bin/ffmpeg -v 9 -loglevel 99 -nostdin -thread_queue_size 512 -i rtmp://127.0.0.1:1935/input/stream1 -i /usr/local/nginx/scripts/images/lowerthird.png -af azmq,volume=2 -c:a aac -filter_complex 'zmq=bind_address=tcp\\\://127.0.0.1\\\:5559,overlay=0:H' -vcodec libx264 -pix_fmt yuv420p -preset veryfast -r 25 -g 50 -b:v 6000k -maxrate 6M -minrate 6M -bufsize 6M -f flv -strict -2 -f flv rtmp://127.0.0.1:1935/distribute/stream1 -y -report
This happens with the latest ffmpeg version (details below). However, I do not see this issue when I use the standard FFmpeg that comes with Ubuntu 16.04's apt-get install (ffmpeg version 2.8.15). To do an aac encode with the Ubuntu standard Ffmpeg, I have to use aac and strict -2. This is why I have used the command above with the latest Ffmpeg version as well, just to standardize the comparsion. Just to reiterate, this issue occurs even when I use a different encoder and without the -strict -2 flags with the latest ffmpeg.
ffmpeg version
ffmpeg version N-92427-gae43235 Copyright (c) 2000-2018 the FFmpeg developers built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.10) 20160609 configuration: --prefix=/root/ffmpeg_build --pkg-config-flags=--static --extra-cflags=-I/root/ffmpeg_build/include --extra-ldflags=-L/root/ffmpeg_build/lib --extra-libs='-lpthread -lm' --enable-gpl --enable-libass --enable-libfdk-aac --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-libzmq --enable-librtmp --enable-network --enable-nonfree libavutil 56. 23.101 / 56. 23.101 libavcodec 58. 39.100 / 58. 39.100 libavformat 58. 22.100 / 58. 22.100 libavdevice 58. 6.100 / 58. 6.100 libavfilter 7. 43.100 / 7. 43.100 libswscale 5. 4.100 / 5. 4.100 libswresample 3. 4.100 / 3. 4.100 libpostproc 55. 4.100 / 55. 4.100
Have attached log output with this ticket. The issue occurs even if I remove all encoding and just do a codec copy as below.
/usr/local/bin/ffmpeg -nostdin -thread_queue_size 512 -i rtmp://127.0.0.1:1935/stream1/input -c copy -f flv rtmp://a.rtmp.youtube.com/live2/[my key] -y
Issue occurs irrespective of destination, whether youtube, facebook or just another location on the same nginx server.
Issue also occurs whether thread_queue_size is 8 or 1024 or 512. It appears that this issue is more frequent in a 4-core system rather than a single core system I was using. Not totally sure about that though.
Attachments (3)
Change History (27)
by , 6 years ago
Attachment: | ffmpeg-20181114-092056.log added |
---|
comment:1 by , 6 years ago
I was having the same problem as regstuff, and found that I was able to resolve my problem with 'av_interleaved_write_frame(): End of file' errors being thrown after about 2 to 3 hours of publishing to an RTMP server by compiling off the 3.4.5 branch, instead of the master. Something must be wrong with the latest version causing this issue?
comment:2 by , 6 years ago
Component: | ffmpeg → undetermined |
---|---|
Keywords: | regression added |
Version: | unspecified → git-master |
Please use git bisect
to find the change introducing the regression.
comment:3 by , 6 years ago
Priority: | normal → important |
---|
I have the same problem while transcoding udp multicast to rtmp via the following command:
ffmpeg -y -hwaccel cuvid -c:v mpeg2_cuvid -resize 1280x720 -deint 2 -drop_second_field 1 -i "udp://235.1.10.1:1234?overrun_nonfatal=1" -aspect 16:9 -vcodec h264_nvenc -b:v 1500k -acodec libfdk_aac -f flv rtmp://127.0.0.1/stream/out
This issue occurs for the newest snapshot from Github. Also, I got the same issue for the 4.1 release version. The same command works well with the 3.4 release version. Unfortunately, we can't use the new features implemented in FFmpeg due to this bug. The problem with RTMP muxing is important for many users
comment:4 by , 6 years ago
Priority: | important → normal |
---|
If the ticket were important you would have used git bisect
to point us to the offending commit.
comment:5 by , 6 years ago
I have this problem on Ubuntu 16.04.5 and Mac with v4.1 when using librtmp
. It works with v3.4.5 on Ubuntu as described above.
I attached a file that shows the issue with the following command (about 4 seconds in):
ffmpeg -re -i issue7547.mp4 -vcodec copy -acodec copy -bsf:a aac_adtstoasc -f flv 'rtmp://a.rtmp.youtube.com/live2/stream-key-here'
The first commit where streaming hangs, git bisect
shows the following:
commit 858db4b01fa2b55ee55056c033054ca54ac9b0fd Author: Daniel Kucera <daniel.kucera@gmail.com> Date: Tue Oct 17 10:29:30 2017 +0200 libavformat: not treat 0 as EOF transfer_func variable passed to retry_transfer_wrapper are h->prot->url_read and h->prot->url_write functions. These need to return EOF or other error properly. In case of returning >= 0, url_read/url_write is retried until error is returned. Signed-off-by: Daniel Kucera <daniel.kucera@gmail.com> :040000 040000 03f37e2b4749f5f1d21a9a528c0a10eef8ff2cf1 6c2edb3a01134a03e79534f6cbb048abb59121d2 M libavformat
The first commit where the error message is correctly displayed, git bisect
shows the following:
commit ed647ab79f9a54d8d3a8e345f6a1643b60b849f4 Author: Timo Rothenpieler <timo@rothenpieler.org> Date: Thu Jul 26 00:37:35 2018 +0200 avformat/librtmp: fix returning EOF from Read/Write Ticket #7052 :040000 040000 964b972c7b3df952e7b634e4a76cdefaa8da3622 200f7038acf21315bee42874f69ae41d6c31315d M libavformat
comment:6 by , 6 years ago
Component: | undetermined → avformat |
---|
comment:7 by , 6 years ago
Priority: | normal → important |
---|
comment:8 by , 6 years ago
Keywords: | librtmp added |
---|
comment:9 by , 6 years ago
I can confirm we are running into the same issue on macOS (10.13.6 and 10.14.3) using 4.1 (installed via Homebrew). So far no issues after downgrading to 3.4.5.
comment:11 by , 6 years ago
Try reverting the first hunk of ed647ab79f9a54d8d3a8e345f6a1643b60b849f4.
The one that changes RTMP_Write. Only RTMP_Read seems to actually do EOF.
comment:12 by , 6 years ago
On another note: Try just not using librtmp. The ffmpeg native rtmp protocol works fine for me.
comment:13 by , 6 years ago
@oromit Reverting the first hunk of that commit causes it to hang at the failing point rather than exit with an error message. The issue first appeared in the other commit in my previous message.
You are correct that using ffmpeg's rtmp instead of librtmp works fine, which explains why I couldn't reproduce on Mac earlier. I updated my comment to note that.
@danman I tried but I'm not familiar with tcpdump. I attached a file and command that reproduces it though.
comment:14 by , 6 years ago
I'd say it's time to get rid of the librtmp support.
I see the same issues here, and the easy fix is to not use librtmp.
comment:15 by , 6 years ago
@ruddell : I don't see the attachment but my random idea is to replace:
if (!ret)
with:
if (!ret && size)
or debug the value of "size" in http://git.videolan.org/?p=ffmpeg.git;a=commitdiff;h=ed647ab79f9a54d8d3a8e345f6a1643b60b849f4
comment:16 by , 6 years ago
@danman Please check the issue's attachments at the top. And the command (copied from my first message): ffmpeg -re -i issue7547.mp4 -vcodec copy -acodec copy -bsf:a aac_adtstoasc -f flv 'rtmp://a.rtmp.youtube.com/live2/stream-key-here'
I tested if (!ret && size)
but it did not cause any difference. The value of size
is 3
at the failing point.
... Write Size: 361 Write Size: 55 Write Size: 32768 Write Size: 3 Write Size: 3
I also tried if (!ret && size > 3)
but it infinitely loops with Write Size: 3
(which is what happens before ed647ab79f9a54d8d3a8e345f6a1643b60b849f4
)
Again, the issue occurs because of the other commit 858db4b01fa2b55ee55056c033054ca54ac9b0fd
in my previous message. The ed647ab79f9a54d8d3a8e345f6a1643b60b849f4
commit simply makes FFMPEG print an error message instead of hanging on RTMP_Write
with Size 3
.
Thanks for looking into this.
comment:17 by , 6 years ago
@ruddell the input file is not enough. I'd like to see the output stream produced. You should be able to capture it with:
tcpdump -i any port port 1935 -w 7547.pcap
comment:18 by , 6 years ago
Thank you for the help with the command. I have attached a pcap file to this issue.
comment:19 by , 6 years ago
To add to the previous comments, I can confirm that the latest version of ffmpeg works fine if I dont use librtmp. Just did a 16 hour test with no issues.
follow-up: 21 comment:20 by , 6 years ago
I can confirm that --disable-librtmp at the ./configure step resolved this issue
comment:21 by , 6 years ago
Replying to ikolesnikov:
I can confirm that --disable-librtmp at the ./configure step resolved this issue
This worked for me when I had the same issue
ffmpeg 4.1 with librtmp + Ubuntu 16.04 streaming to nginx-rtmp = error described in the original post
ffmpeg 4.1 with librtmp disabled is fine
comment:22 by , 5 years ago
No need to use --disable-librtmp
as librtmp support is not automatically enabled and is only enabled if you use --enable-librtmp
. Refer to ./configure --help
to see what is automatically enabled.
So just omit --enable-librtmp
.
comment:23 by , 4 years ago
Cc: | added |
---|
hi all:
I have the same problem.
reproduce steps:
- send the stream some seconds (about 10 seconds).
- stop the stream 5~10 seconds
- resend the stream, the ffmpeg exit with:
[flv @ 0x3e733c0] Failed to update header with correct duration.
[flv @ 0x3e733c0] Failed to update header with correct filesize.
Error writing trailer of rtmp://xxxxx
I have try:
--disable-librtmp but no effect, the problem cause again.
ffmpeg version:
================================================================================
ffmpeg version 4.2.2 Copyright (c) 2000-2019 the FFmpeg developers
built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.12) 20160609
configuration: --prefix=/usr/sourcecode/ffmpegout --pkg-config-flags=--static -- extra-cflags=-I/usr/sourcecode/ffmpegout/include --extra-ldflags=-L/usr/sourceco de/ffmpegout/lib --enable-gpl --enable-libass --enable-libfdk-aac --enable-libfr eetype --enable-libmp3lame --enable-libtheora --enable-libvorbis --enable-nonfre e --enable-shared --enable-postproc --enable-zlib --enable-libxvid --enable-libo pencore-amrnb --enable-libopencore-amrwb --enable-version3 --enable-libx264 --di sable-debug
libavutil 56. 31.100 / 56. 31.100
libavcodec 58. 54.100 / 58. 54.100
libavformat 58. 29.100 / 58. 29.100
libavdevice 58. 8.100 / 58. 8.100
libavfilter 7. 57.100 / 7. 57.100
libswscale 5. 5.100 / 5. 5.100
libswresample 3. 5.100 / 3. 5.100
libpostproc 55. 5.100 / 55. 5.100
comment:24 by , 4 years ago
while enable-librtmp:
WriteN, RTMP send error 32 (129 bytes)
RTMP_SendPacket: fd=5, size=47
0000: 43 00 00 00 00 00 2f 14 C...../.
0000: 02 00 0b 46 43 55 6e 70 75 62 6c 69 73 68 00 40 ...FCUnpublish.@
0010: 1c 00 00 00 00 00 00 05 02 00 14 31 33 35 35 35 ...........13555
0020: 30 30 30 30 30 33 5f 31 5f 30 5f 31 30 37 38 000003_1_0_1078
WriteN, RTMP send error 32 (55 bytes)
RTMP_SendPacket: fd=-1, size=34
0000: 43 00 00 00 00 00 22 14 C.....".
0000: 02 00 0c 64 65 6c 65 74 65 53 74 72 65 61 6d 00 ...deleteStream.
0010: 3f f0 00 00 00 00 00 00 05 00 3f f0 00 00 00 00 ?.........?.....
0020: 00 00 ..
WriteN, RTMP send error 9 (42 bytes)
[rtmp @ 0x128d740] rtmp_write:-1
Ffmpeg Log File