Memory leaks in aac decode

[Matt Wolenetz]

Summary of the bug:

There are 2 leaks hit for the attached “null1.m4a” file.   See attached “aac_leaks.log” for detail.

I hit these in current master (f51aa92b0f165a24c4bd8c3b0b85c159048195d6).  Bisecting shows at least one of these leaks may have pre-existed and have leaked more memory since 80e9e63c946660304fc65fa8141ccfdbe4d196d1 (lavc decoders: work with refcounted frames).

How to reproduce:

./configure --disable-optimizations --extra-cflags='-fno-inline -fno-omit-frame-pointer -fno-builtin -fno-optimize-sibling-calls' --enable-debug --enable-static --disable-shared && make clean && make -j32 && ../../trunk/src/third_party/valgrind/linux_x64/bin/valgrind --tool=memcheck --num-callers=30 --gen-suppressions=all --demangle=no --leak-check=yes --show-possibly-lost=no ./ffmpeg_g -i null1.m4a -f null - 2>&1 | tee aac_leaks.log

Patches should be submitted to the ffmpeg-devel mailing list and not this bug tracker.

[Hendrik]

I don't have Valgrind handy, but can you check if this possible solves it?

[Matt Wolenetz]

Thanks! I will try that soon today. If it works, remember this isn't a forum for submitting the patch itself (ffmpeg-devel is).

[Hendrik]

Of course, but helps to know if the patch works before officially submitting it. :)

[Carl Eugen Hoyos]

This is a duplicate of ticket #2095, I will update the information there next week.
(The valgrind output is outdated.)

The patch does not fix the leak here.

[Carl Eugen Hoyos]

Apparently not (only) a duplicate of ticket #2095.

[Carl Eugen Hoyos]

$ valgrind --leak-check=full ffmpeg_g -i null1.m4a -f null -
==7255== Memcheck, a memory error detector
==7255== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al.
==7255== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info
==7255== Command: ffmpeg_g -i null1.m4a -f null -
==7255==
ffmpeg version N-52789-g2d8dddd Copyright (c) 2000-2013 the FFmpeg developers
  built on May  5 2013 18:25:21 with gcc 4.7 (SUSE Linux)
  configuration: --enable-gpl --disable-indev=jack
  libavutil      52. 29.100 / 52. 29.100
  libavcodec     55.  7.100 / 55.  7.100
  libavformat    55.  4.101 / 55.  4.101
  libavdevice    55.  0.100 / 55.  0.100
  libavfilter     3. 63.101 /  3. 63.101
  libswscale      2.  2.100 /  2.  2.100
  libswresample   0. 17.102 /  0. 17.102
  libpostproc    52.  3.100 / 52.  3.100
[aac @ 0x64ba4a0] decode_band_types: Input buffer exhausted before END element found
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'null1.m4a':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: M4A mp42isom
    creation_time   : 2010-04-20 23:04:52
    encoder         : Nero AAC codec / 1.5.3.0
  Duration: 00:00:02.88, start: 0.000000, bitrate: 125 kb/s
    Chapter #0.0: start 0.105941, end 2.879267
    Metadata:
      title           :
    Stream #0:0(und): Audio: aac (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 88 kb/s
    Metadata:
      creation_time   : 2010-04-20 23:04:52
      handler_name    : Sound Media Handler
Output #0, null, to 'pipe:':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: M4A mp42isom
    encoder         : Lavf55.4.101
    Chapter #0.0: start 0.105941, end 2.879267
    Metadata:
      title           :
    Stream #0:0(und): Audio: pcm_s16le, 44100 Hz, stereo, s16, 1411 kb/s
    Metadata:
      creation_time   : 2010-04-20 23:04:52
      handler_name    : Sound Media Handler
Stream mapping:
  Stream #0:0 -> #0:0 (aac -> pcm_s16le)
Press [q] to stop, [?] for help
Multiple frames in a packet from stream 0
[aac @ 0x64ba4a0] Number of scalefactor bands in group (15) exceeds limit (14).
Error while decoding stream #0:0: Invalid data found when processing input
envelope scalefactor overflow in dequant

...

[aac @ 0x64ba4a0] invalid band type
Error while decoding stream #0:0: Operation not permitted
size=N/A time=00:00:00.00 bitrate=N/A
video:0kB audio:0kB subtitle:0 global headers:0kB muxing overhead -inf%
Output file is empty, nothing was encoded (check -ss / -t / -frames parameters if used)
==7255==
==7255== HEAP SUMMARY:
==7255==     in use at exit: 33,088 bytes in 13 blocks
==7255==   total heap usage: 1,659 allocs, 1,646 frees, 27,746,641 bytes allocated
==7255==
==7255== 33,088 (48 direct, 33,040 indirect) bytes in 2 blocks are definitely lost in loss record 5 of 5
==7255==    at 0x4C290FE: memalign (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==7255==    by 0x4C291A7: posix_memalign (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==7255==    by 0xBDF371: av_mallocz (mem.c:93)
==7255==    by 0xBD33E7: av_buffer_alloc (buffer.c:47)
==7255==    by 0xBD3BFB: av_buffer_pool_get (buffer.c:305)
==7255==    by 0x9B2B6D: audio_get_buffer (utils.c:491)
==7255==    by 0x9B42FE: get_buffer_internal (utils.c:815)
==7255==    by 0x9B4825: ff_get_buffer (utils.c:827)
==7255==    by 0xA988AF: aac_decode_frame_int (aacdec.c:197)
==7255==    by 0xA9ADC1: aac_decode_frame (aacdec.c:2690)
==7255==    by 0x9B68FB: avcodec_decode_audio4 (utils.c:2071)
==7255==    by 0x469197: decode_audio (ffmpeg.c:1500)
==7255==
==7255== LEAK SUMMARY:
==7255==    definitely lost: 48 bytes in 2 blocks
==7255==    indirectly lost: 33,040 bytes in 11 blocks
==7255==      possibly lost: 0 bytes in 0 blocks
==7255==    still reachable: 0 bytes in 0 blocks
==7255==         suppressed: 0 bytes in 0 blocks
==7255==
==7255== For counts of detected and suppressed errors, rerun with: -v
==7255== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 2 from 2)

[Matt Wolenetz]

With current git-master, I've done some investigating and it looks like the ac->frame buffers that frame_configure_elements() gets from ff_get_buffer() are leaked when the next log message is "Multiple frames in a packet from stream 0". Furthermore, one of those frames contains references apparently to a couple new frames that frame_configure_elements() on its next call gets from ff_get_buffer(), even though those latter frames are eventually unref'ed() once. I hope this helps narrow down the search for a fix. Details ("[LEAKED...]" comments are mine, line numbers may be off slightly due to private debug log msg additions):

==31122== 33,088 (+33,088) (48 (+48) direct, 33,040 (+33,040) indirect) bytes in 2 (+2) blocks are definitely lost in loss record 138 of 138
4270 ==31122== at 0x4C2B292: posix_memalign (vg_replace_malloc.c:878)
4271 ==31122== by 0xD3BA31: av_malloc (mem.c:93)
4272 ==31122== by 0xD3BB91: av_mallocz (mem.c:207)
4273 ==31122== by 0xD2F16D: av_buffer_create (buffer.c:49)
4274 ==31122== by 0xD2F22B: av_buffer_alloc (buffer.c:76)
4275 ==31122== by 0xD2F98F: pool_alloc_buffer (buffer.c:318)
4276 ==31122== by 0xD2FB03: av_buffer_pool_get (buffer.c:356)
4277 ==31122== by 0xA1E050: audio_get_buffer (utils.c:491)
4278 ==31122== by 0xA1E71D: avcodec_default_get_buffer2 (utils.c:602)
4279 ==31122== by 0xA1F09D: get_buffer_internal (utils.c:815)
4280 ==31122== by 0xA1F100: ff_get_buffer (utils.c:827)
4281 ==31122== by 0xB69A12: frame_configure_elements (aacdec.c:218)
4282 ==31122== by 0xB72F72: aac_decode_frame_int (aacdec.c:2545)
4283 ==31122== by 0xB73590: aac_decode_frame (aacdec.c:2729)
4284 ==31122== by 0xA22F6A: avcodec_decode_audio4 (utils.c:2071)
4285 ==31122== by 0x417A93: decode_audio (ffmpeg.c:1500)
4286 ==31122== by 0x41940A: output_packet (ffmpeg.c:1833)
4287 ==31122== by 0x41EDEF: process_input (ffmpeg.c:3042)
4288 ==31122== by 0x41F16C: transcode_step (ffmpeg.c:3138)
4289 ==31122== by 0x41F2A5: transcode (ffmpeg.c:3195)
4290 ==31122== by 0x41F7E7: main (ffmpeg.c:3378)
4291 ==31122== 0x63ADA80[24]
4292 ==31122== 0x63AD8E0[40] indirect loss record 91
4293 ==31122== 0x63AB780[8192] indirect loss record 137 [LEAKED, from ff_get_buffer just prior to the "Multiple frames in a packet from stream 0"]
4294 ==31122== 0x63ADC00[40] indirect loss record 113
4295 ==31122== 0x63AB600[32] indirect loss record 69
4296 ==31122== 0x63B2800[40] indirect loss record 113
4297 ==31122== 0x63B0380[8192] indirect loss record 137 [LEAKED, from ff_get_buffer just after the "Multiple frames in a packet from stream 0", leaks even though eventually frame unref'ed once by frame_configure_elements()]
4298 ==31122== 0x63B4E00[40] indirect loss record 113
4299 ==31122== 0x63B2980[8192] indirect loss record 137 [LEAKED, ditto of 0x63B0380 frame]
4300 ==31122== 0x63B0080[24]
4301 ==31122== 0x63AFEE0[40] indirect loss record 91
4302 ==31122== 0x63ADD80[8192] indirect loss record 137 [LEAKED, from ff_get_buffer just prior to the "Multiple frames in a packet from stream 0"]
4303 ==31122== 0x63B0200[40] indirect loss record 113

[Matt Wolenetz]

Attached my verbose debug log that shows better format of monitor block_list output ("leak_debug.log").

[Matt Wolenetz]

Valgrind gdbserver attached session output showing block_list details of leaked memory.

[Matt Wolenetz]

Enabling -v debug shows many of those "Multiple frames in a packet" messages. Just the first one is correlated with the leaks. Also, there is a "skip whole frame, skip left: 2624" message output in debug mode that also only occurs adjacent to this first "Multiple frames in a packet" message but never elsewhere in this repro with debug logging.

[Matt Wolenetz]

It looks like I have a potential fix for this remaining leak. I'll send a patch to ffmpeg-devel for review because I don't know if my patch would cause use-after-free for other media decodes.

[Matt Wolenetz]

See ffmpeg-devel@ thread "[PATCH] Fix leak by dereferencing audio frame when side data causes whole frame skip" for review of proposed fix that, along with d18341fb1121332056aecc00096159df16d01107, fixes memleaks for the attached null1.m4a sample.

[Carl Eugen Hoyos]

Fixed by you in 004779c
For future patches: If a patch fixes a ticket, please add "Fixes ticket #xxxx" to the commit message.

[Matt Wolenetz]

Thank you for following up. I had that in the ffmpeg-devel message, but mistakenly not in the commit message in the patch I submitted there, hence the ticket reference was dropped on eventual commit by Michael.