Opened 7 years ago
Last modified 6 years ago
#6379 new defect
vaapi_encode_check_config invalid free
Reported by: | serafean | Owned by: | |
---|---|---|---|
Priority: | important | Component: | avcodec |
Version: | 3.2.4 | Keywords: | crash vaapi |
Cc: | Blocked By: | ||
Blocking: | Reproduced by developer: | no | |
Analyzed by developer: | no |
Description
Invalid free in vaapi_encode_check_config ( vaapi_encode_config_attributes in 3.3 branch - manually checked source code )
How to reproduce:
% MALLOC_CHECK_=2 ffmpeg -loglevel debug -hwaccel vaapi -vaapi_device /dev/dri/renderD128 -i Elephants_Dream_HD.avi -vf format=nv12,hwupload -map 0:0 -map 0:1 -y -f matroska -bf 0 -c:v h264_vaapi ~/test.mkv 3.2.4 built on Gentoo
(gdb) bt #0 0x00007f0fbeb1eeb8 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:54 #1 0x00007f0fbeb2044a in __GI_abort () at abort.c:89 #2 0x00007f0fbeb63890 in malloc_printerr (action=<optimized out>, str=0x7f0fbec55c27 "free(): invalid pointer", ptr=<optimized out>, ar_ptr=<optimized out>) at malloc.c:5008 #3 0x00007f0fc01e88b2 in vaapi_encode_check_config (avctx=0x55e6562a3620) at src/libavcodec/vaapi_encode.c:1024 #4 ff_vaapi_encode_init (avctx=0x55e6562a3620, type=<optimized out>) at src/libavcodec/vaapi_encode.c:1076 #5 0x00007f0fc06353f0 in avcodec_open2 (avctx=0x55e6562a3620, codec=0x7f0fc1043cc0 <ff_h264_vaapi_encoder>, options=0x55e65624f888) at src/libavcodec/utils.c:1608 #6 0x000055e655ab58ca in init_output_stream (error_len=1024, error=0x7ffde9fa30e0 "", ost=0x55e65624f740) at src/ffmpeg.c:3024 #7 transcode_init () at src/ffmpeg.c:3482 #8 0x000055e655a98352 in transcode () at src/ffmpeg.c:4358 #9 main (argc=23, argv=0x7ffde9fa3a48) at src/ffmpeg.c:4592
The issue is that every "goto fail" tries to free both "profiles" and "entrypoints", when entrypoints might not even be allocated yet.
Change History (6)
comment:1 by , 7 years ago
comment:2 by , 7 years ago
Ah, my bad, I forgot about the free behaviour.
Crashes.
with MALLOC_CHECK_=1 segfault :
#0 _int_malloc (av=av@entry=0x7f8429308aa0 <main_arena>, bytes=bytes@entry=9) at malloc.c:3414 #1 0x00007f8428fe635b in malloc_check (sz=sz@entry=8, caller=caller@entry=0x0) at hooks.c:295 #2 0x00007f8428fe6d86 in realloc_check (oldmem=0x0, bytes=8, caller=<optimized out>) at hooks.c:355 #3 0x00007f8429a80c4f in av_frame_new_side_data (frame=frame@entry=0x563b5c49d7a0, type=AV_FRAME_DATA_MATRIXENCODING, size=4) at src/libavutil/frame.c:634 #4 0x00007f8429a80f40 in frame_copy_props (dst=dst@entry=0x563b5c49d7a0, src=0x563b5c4af0e0, force_copy=force_copy@entry=1) at src/libavutil/frame.c:339 #5 0x00007f8429a810a9 in av_frame_copy_props (dst=dst@entry=0x563b5c49d7a0, src=<optimized out>) at src/libavutil/frame.c:591 #6 0x00007f842be92141 in ff_filter_frame_needs_framing (frame=0x563b5c4af0e0, link=0x563b5c36c6c0) at src/libavfilter/avfilter.c:1162 #7 ff_filter_frame (link=0x563b5c36c6c0, frame=0x563b5c4af0e0) at src/libavfilter/avfilter.c:1230 #8 0x00007f842be90d9a in ff_filter_frame_framed (link=link@entry=0x563b5c36c540, frame=0x563b5c4af0e0) at src/libavfilter/avfilter.c:1134 #9 0x00007f842be920c0 in ff_filter_frame (link=0x563b5c36c540, frame=0x563b5c4af0e0) at src/libavfilter/avfilter.c:1232 #10 0x00007f842be90d9a in ff_filter_frame_framed (link=link@entry=0x563b5c36c2c0, frame=0x563b5c4af0e0) at src/libavfilter/avfilter.c:1134 #11 0x00007f842be920c0 in ff_filter_frame (link=link@entry=0x563b5c36c2c0, frame=0x563b5c4af0e0) at src/libavfilter/avfilter.c:1232 #12 0x00007f842be9658f in request_frame (link=0x563b5c36c2c0) at src/libavfilter/buffersrc.c:450 #13 0x00007f842be96244 in av_buffersrc_add_frame_internal (ctx=ctx@entry=0x563b5c36b840, frame=frame@entry=0x563b5c45db60, flags=flags@entry=4) at src/libavfilter/buffersrc.c:239 #14 0x00007f842be967ed in av_buffersrc_add_frame_flags (ctx=0x563b5c36b840, frame=0x563b5c45db60, flags=4) at src/libavfilter/buffersrc.c:164 #15 0x0000563b5c065a7d in decode_audio (got_output=0x7ffdb8e80c0c, pkt=0x7ffdb8e80c30, ist=0x563b5c342ee0) at src/ffmpeg.c:2164 #16 process_input_packet (ist=<optimized out>, pkt=0x7ffdb8e80f00, no_eof=0) at src/ffmpeg.c:2466 #17 0x0000563b5c046daa in process_input (file_index=<optimized out>) at src/ffmpeg.c:4245 #18 transcode_step () at src/ffmpeg.c:4333 #19 transcode () at src/ffmpeg.c:4387 #20 main (argc=<optimized out>, argv=<optimized out>) at src/ffmpeg.c:4592
Without MALLOC_CHECK_, sometimes SIGABRT:
*** Error in `ffmpeg': corrupted double-linked list: 0x000055ad35f86170 *** ======= Backtrace: ========= /lib64/libc.so.6(+0x72a07)[0x7f784ac7da07] /lib64/libc.so.6(+0x78866)[0x7f784ac83866] /lib64/libc.so.6(+0x78c21)[0x7f784ac83c21] /lib64/libc.so.6(+0x7a5d2)[0x7f784ac855d2] /lib64/libc.so.6(__libc_malloc+0x63)[0x7f784ac876f3] /usr/lib64/va/drivers/r600_drv_video.so(+0x210ca)[0x7f783bf140ca] /usr/lib64/va/drivers/r600_drv_video.so(+0x22fa2)[0x7f783bf15fa2] /usr/lib64/libavutil.so.55(+0x2072e)[0x7f784b72372e] /usr/lib64/libavutil.so.55(+0x20a77)[0x7f784b723a77] /usr/lib64/libavutil.so.55(av_hwframe_transfer_data+0xb7)[0x7f784b722ec7] /usr/lib64/libavfilter.so.6(+0x10ef89)[0x7f784dba1f89] /usr/lib64/libavfilter.so.6(+0x9dd9a)[0x7f784db30d9a] /usr/lib64/libavfilter.so.6(+0x9f0c0)[0x7f784db320c0] /usr/lib64/libavfilter.so.6(+0x9dd9a)[0x7f784db30d9a] /usr/lib64/libavfilter.so.6(+0x9f0c0)[0x7f784db320c0] /usr/lib64/libavfilter.so.6(+0x13d466)[0x7f784dbd0466] /usr/lib64/libavfilter.so.6(+0x9dd9a)[0x7f784db30d9a] /usr/lib64/libavfilter.so.6(+0x9f0c0)[0x7f784db320c0] /usr/lib64/libavfilter.so.6(+0xa358f)[0x7f784db3658f] /usr/lib64/libavfilter.so.6(+0xa3244)[0x7f784db36244] /usr/lib64/libavfilter.so.6(av_buffersrc_add_frame_flags+0xb5)[0x7f784db367ed] ffmpeg(+0x2bd05)[0x55ad34c4cd05] ffmpeg(+0xcdaa)[0x55ad34c2ddaa] /lib64/libc.so.6(__libc_start_main+0xfc)[0x7f784ac2b7cc] ffmpeg(+0xea59)[0x55ad34c2fa59]
running it through Valgrind throws out SIGILL.
SIGSEGV in malloc is the most common.
MALLOC_CHECK_=2 is the only one which is deterministic, so I thought that could be it...
I'll try with ffmpeg 3.3 tomorrow.
Do you think it could be in the r600 driver?
follow-up: 5 comment:3 by , 7 years ago
Keywords: | crash vaapi added |
---|---|
Priority: | normal → important |
Please test current FFmpeg git head.
comment:4 by , 7 years ago
ffmpeg version : N-85962-g164e277
valgrind still raises SIGILL
MALLOC_CHECK_=2 still crashes with
(gdb) bt #0 0x00007ffabdfc6eb8 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:54 #1 0x00007ffabdfc844a in __GI_abort () at abort.c:89 #2 0x00007ffabe00b890 in malloc_printerr (action=<optimized out>, str=0x7ffabe0fdc27 "free(): invalid pointer", ptr=<optimized out>, ar_ptr=<optimized out>) at malloc.c:5008 #3 0x00007ffabf470411 in vaapi_encode_config_attributes (avctx=0x55ef16408aa0) at src/libavcodec/vaapi_encode.c:1107 #4 ff_vaapi_encode_init (avctx=0x55ef16408aa0) at src/libavcodec/vaapi_encode.c:1377 #5 0x00007ffabf8d78f9 in avcodec_open2 (avctx=0x55ef16408aa0, codec=0x7ffac0304ce0 <ff_h264_vaapi_encoder>, options=0x55ef163a67f8) at src/libavcodec/utils.c:1020 #6 0x000055ef142c8dde in init_output_stream (ost=<optimized out>, error=0x7fff49d94c20 "", error_len=1024) at src/ffmpeg.c:3438 #7 0x000055ef142ca9c1 in reap_filters (flush=0) at src/ffmpeg.c:1443 #8 0x000055ef142acd94 in transcode_step () at src/ffmpeg.c:4522 #9 transcode () at src/ffmpeg.c:4566 #10 main (argc=<optimized out>, argv=<optimized out>) at src/ffmpeg.c:477
The other two cases appear to work (albeit slowly, but that might be an r600 issue).
MALLOC_CHECK_=3 :
*** Error in `ffmpeg': free(): invalid pointer: 0x0000560c61308b20 *** ======= Backtrace: ========= /lib64/libc.so.6(+0x72a07)[0x7fea668a5a07] /lib64/libc.so.6(+0x78866)[0x7fea668ab866] /usr/lib64/libavcodec.so.57(+0xad411)[0x7fea67d10411] /usr/lib64/libavcodec.so.57(avcodec_open2+0x879)[0x7fea681778f9] ffmpeg(+0x28dde)[0x560c6094ddde] ffmpeg(+0x2a9c1)[0x560c6094f9c1] ffmpeg(+0xcd94)[0x560c60931d94] /lib64/libc.so.6(__libc_start_main+0xfc)[0x7fea668537cc] ffmpeg(+0xec69)[0x560c60933c69] ======= Memory map: ========
comment:5 by , 6 years ago
Replying to cehoyos:
where can I get Elephants_Dream_HD.avi , I try to reproduce this issue in master
comment:6 by , 6 years ago
Not sure why you are asking me but here is a working link:
https://www.mediaspip.net/IMG/avi/Elephants_Dream_HD.avi
What is the actual error that you get here?
I don't see anything wrong with the entrypoints variable - it's initialised to NULL and then possibly overwritten by the return value of av_malloc_array(). Both of those are always valid things to pass to free().