Memory leak in ff_read_packet when decoding udp mpegts multicast stream

[Thomas Hutschenreuther]

Summary of the bug:

I am receiving 4 udp multicast streams on different ports, which each contain 4 programs.
I transcode one program out of each of those using one ffmpeg instance.
Everything works fine until at some random point in time, the memory usage of ffmpeg starts to grow constantly.

It seams as if at this point in time, the mpegts demuxer reports a bogus new stream and sets its request_probe property to 1.

Now since this stream is bogus there won't be any subsequent packets for this stream.

The problem is, that this packet will be put to the raw_packet_buffer in ff_read_packet.

Now the loop in ff_read_packet has no chance of ever terminating because each new packet will be put on the packet buffer but the first packet will never be popped since probing will never be done because no new packet for the stream to be probed will arrive.

Thus I suggest the following patch

diff --git a/libavformat/utils.c b/libavformat/utils.c
index 284cb9f..f998b0e 100644
--- a/libavformat/utils.c
+++ b/libavformat/utils.c
@@ -720,10 +720,12 @@ int ff_read_packet(AVFormatContext *s, AVPacket *pkt)
         if (pktl) {
             *pkt = pktl->pkt;
             st = s->streams[pkt->stream_index];
-            if(st->request_probe <= 0){
+            if(st->request_probe <= 0 || s->raw_packet_buffer_remaining_size <= 0){
                 s->raw_packet_buffer = pktl->next;
                 s->raw_packet_buffer_remaining_size += pkt->size;
                 av_free(pktl);
+                if(s->raw_packet_buffer_remaining_size <= 0)
+                    av_log(s, AV_LOG_WARNING, "probing stream %d failed", st->index);
                 return 0;
             }
         }

How to reproduce:
The reproduction is a bit tricky since I only encountered this problem with multicast udp mpegts streams

I have attached a complete log file.
The memory growth starts after the entry in line 28650.

When running ffmpeg in gdb and interrupting after the memory growth has started, extremely large negative values for s->raw_buffer_remaining_size can be observed.

Since this is my first bug report here, I want to apologize for any formal mistakes.

[Thomas Hutschenreuther]

log-file generated while observing the bug

[Carl Eugen Hoyos]

Please post patches to ffmpeg-devel, they receive more attention there.

[Michael Niedermayer]

The check for printing the warning is after the raw_packet_buffer_remaining_size is changed

[Thomas Hutschenreuther]

Replying to michael:

You are right...better check on the other condition (st->request_probe > 0)
But that also leaves some problems open.
Should the request_probe property of the corresponding stream be negated?
Should the stream be closed / is it possible to terminate a stream once opened?
From what I see in the corresponding sources (libavformat/utils.c and libavformat/mpegts.c), there only seems to be functionality to spawn new streams and not to close them.
Maybe one should also free the packet on reporting probing failure.

[Dave]

Any word on this issue? This memory leak is pretty severe and basically makes UDP input useless for long data reception. I would guess it's probably fine for short files where you restart ffmpeg before the memory leak becomes severe enough to take the process down.

Any workarounds? It would be difficult for me to roll back to an earlier ffmpeg version since I use Xuggler and bundled ffmpeg libraries. In my case the memory leak just runs the jvm process up to machine installed memory and then crashes the jvm.

[Thomas Hutschenreuther]

For me this only occurs in correlation with #1510.
Setting ts->auto_guess to 0 in mpegts.c:1954 (the name of the function is mpegts_read_header) worked in my case and I did not see any bad side effects after several weeks of permanently decoding udp streams I generate from dvb signals with another tool using the libraries compiled this way.

[MB]

I've experienced an apparently identical issue with recent ffmpeg builds (Zeranoe, 2015-02-03).
Would be useful to know at least a workaround for the issue, of course a fix would be great.

[Marton Balint]

Is this still reproducable?

[Carl Eugen Hoyos]

This would need valgrind output to be a valid ticket.