#2675 closed defect (fixed)
postproc crashes with -fstack-protector-all
| Reported by: | Carl Eugen Hoyos | Owned by: | Michael Niedermayer |
|---|---|---|---|
| Priority: | normal | Component: | postproc |
| Version: | git-master | Keywords: | crash SIGSEGV gcc |
| Cc: | Blocked By: | ||
| Blocking: | Reproduced by developer: | no | |
| Analyzed by developer: | no |
Description
As reported in http://thread.gmane.org/gmane.comp.video.ffmpeg.user/46215/focus=46220
Only crashes with MMX2 and SSE2, works fine for C-only and MMX
$ ffmpeg -f lavfi -i testsrc -vf pp=dr -f null -
ffmpeg version N-54036-g6c4516d Copyright (c) 2000-2013 the FFmpeg developers
built on Jun 16 2013 15:48:02 with gcc 4.7 (SUSE Linux)
configuration: --enable-gpl --disable-indev=jack --extra-cflags=-fstack-protector-all
libavutil 52. 35.101 / 52. 35.101
libavcodec 55. 16.100 / 55. 16.100
libavformat 55. 8.102 / 55. 8.102
libavdevice 55. 2.100 / 55. 2.100
libavfilter 3. 77.101 / 3. 77.101
libswscale 2. 3.100 / 2. 3.100
libswresample 0. 17.102 / 0. 17.102
libpostproc 52. 3.100 / 52. 3.100
Input #0, lavfi, from 'testsrc':
Duration: N/A, start: 0.000000, bitrate: N/A
Stream #0:0: Video: rawvideo (RGB[24] / 0x18424752), rgb24, 320x240 [SAR 1:1 DAR 4:3], 25 tbr, 25 tbn, 25 tbc
Output #0, null, to 'pipe:':
Metadata:
encoder : Lavf55.8.102
Stream #0:0: Video: rawvideo (444P / 0x50343434), yuv444p, 320x240 [SAR 1:1 DAR 4:3], q=2-31, 200 kb/s, 90k tbn, 25 tbc
Stream mapping:
Stream #0:0 -> #0:0 (rawvideo -> rawvideo)
Press [q] to stop, [?] for help
Segmentation fault
(gdb) r -f lavfi -i testsrc -vf pp=dr -f null -
Starting program: ffmpeg_g -f lavfi -i testsrc -vf pp=dr -f null -
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
ffmpeg version N-54036-g6c4516d Copyright (c) 2000-2013 the FFmpeg developers
built on Jun 16 2013 15:48:02 with gcc 4.7 (SUSE Linux)
configuration: --enable-gpl --disable-indev=jack --extra-cflags=-fstack-protector-all
libavutil 52. 35.101 / 52. 35.101
libavcodec 55. 16.100 / 55. 16.100
libavformat 55. 8.102 / 55. 8.102
libavdevice 55. 2.100 / 55. 2.100
libavfilter 3. 77.101 / 3. 77.101
libswscale 2. 3.100 / 2. 3.100
libswresample 0. 17.102 / 0. 17.102
libpostproc 52. 3.100 / 52. 3.100
[New Thread 0x7ffff59e7700 (LWP 15836)]
[New Thread 0x7ffff51e6700 (LWP 15837)]
[New Thread 0x7ffff49e5700 (LWP 15838)]
[New Thread 0x7ffff41e4700 (LWP 15839)]
[New Thread 0x7ffff39e3700 (LWP 15840)]
[New Thread 0x7ffff31e2700 (LWP 15841)]
[New Thread 0x7ffff29e1700 (LWP 15842)]
[New Thread 0x7ffff21e0700 (LWP 15843)]
[New Thread 0x7ffff19df700 (LWP 15844)]
Input #0, lavfi, from 'testsrc':
Duration: N/A, start: 0.000000, bitrate: N/A
Stream #0:0: Video: rawvideo (RGB[24] / 0x18424752), rgb24, 320x240 [SAR 1:1 DAR 4:3], 25 tbr, 25 tbn, 25 tbc
[New Thread 0x7ffff11de700 (LWP 15845)]
[New Thread 0x7ffff09dd700 (LWP 15846)]
[New Thread 0x7ffff01dc700 (LWP 15847)]
[New Thread 0x7fffef9db700 (LWP 15848)]
[New Thread 0x7fffef1da700 (LWP 15849)]
[New Thread 0x7fffee9d9700 (LWP 15850)]
[New Thread 0x7fffee1d8700 (LWP 15851)]
[New Thread 0x7fffed9d7700 (LWP 15852)]
[New Thread 0x7fffed1d6700 (LWP 15853)]
Output #0, null, to 'pipe:':
Metadata:
encoder : Lavf55.8.102
Stream #0:0: Video: rawvideo (444P / 0x50343434), yuv444p, 320x240 [SAR 1:1 DAR 4:3], q=2-31, 200 kb/s, 90k tbn, 25 tbc
Stream mapping:
Stream #0:0 -> #0:0 (rawvideo -> rawvideo)
Press [q] to stop, [?] for help
Program received signal SIGSEGV, Segmentation fault.
0x0000000000bd4cfd in dering_SSE2 (
src=0x17c8ec0 '\020' <repeats 40 times>, 'Q' <repeats 38 times>"\252, \252", 'j' <repeats 40 times>, ')' <repeats 40 times>"\322, \322\322\322\322\322\322\322\322\322\322\322\322\322\322\322\322\322\322\322\322\322\322\322\322\322\322\322\322\322\322\322\322\322\322\322\322\322\322", <incomplete sequence \322>...,
src@entry=<error reading variable: Cannot access memory at address 0x10001028>, stride=320,
stride@entry=<error reading variable: Cannot access memory at address 0x10001028>, c=0x17c9500,
c@entry=<error reading variable: Cannot access memory at address 0x10001028>) at libpostproc/postprocess_template.c:1094
1094 __asm__ volatile(
(gdb) bt
#0 0x0000000000bd4cfd in dering_SSE2 (
src=0x17c8ec0 '\020' <repeats 40 times>, 'Q' <repeats 38 times>"\252, \252", 'j' <repeats 40 times>, ')' <repeats 40 times>"\322, \322\322\322\322\322\322\322\322\322\322\322\322\322\322\322\322\322\322\322\322\322\322\322\322\322\322\322\322\322\322\322\322\322\322\322\322\322\322", <incomplete sequence \322>...,
src@entry=<error reading variable: Cannot access memory at address 0x10001028>, stride=320,
stride@entry=<error reading variable: Cannot access memory at address 0x10001028>, c=0x17c9500,
c@entry=<error reading variable: Cannot access memory at address 0x10001028>) at libpostproc/postprocess_template.c:1094
Cannot access memory at address 0x10001028
(gdb) disass $pc-32,$pc+32
Dump of assembler code from 0xbd4cdd to 0xbd4d1d:
0x0000000000bd4cdd <dering_SSE2+173>: pshufw $0xf9,%mm6,%mm4
0x0000000000bd4ce1 <dering_SSE2+177>: pmaxub %mm4,%mm6
0x0000000000bd4ce4 <dering_SSE2+180>: pshufw $0xfe,%mm6,%mm4
0x0000000000bd4ce8 <dering_SSE2+184>: pmaxub %mm4,%mm6
0x0000000000bd4ceb <dering_SSE2+187>: movq %mm6,%mm0
0x0000000000bd4cee <dering_SSE2+190>: psubb %mm7,%mm6
0x0000000000bd4cf1 <dering_SSE2+193>: push %rsp
0x0000000000bd4cf2 <dering_SSE2+194>: movd %mm6,%esp
0x0000000000bd4cf5 <dering_SSE2+197>: cmp 0xe3d260,%spl
=> 0x0000000000bd4cfd <dering_SSE2+205>: pop %rsp
0x0000000000bd4cfe <dering_SSE2+206>: jb 0xbd5291 <dering_SSE2+1633>
0x0000000000bd4d04 <dering_SSE2+212>: pavgb %mm0,%mm7
0x0000000000bd4d07 <dering_SSE2+215>: punpcklbw %mm7,%mm7
0x0000000000bd4d0a <dering_SSE2+218>: punpcklbw %mm7,%mm7
0x0000000000bd4d0d <dering_SSE2+221>: punpcklbw %mm7,%mm7
0x0000000000bd4d10 <dering_SSE2+224>: movq %mm7,(%rsp)
0x0000000000bd4d14 <dering_SSE2+228>: movq (%rdi),%mm0
0x0000000000bd4d17 <dering_SSE2+231>: movq %mm0,%mm1
0x0000000000bd4d1a <dering_SSE2+234>: movq %mm0,%mm2
End of assembler dump.
(gdb) info all-register
rax 0x17c9000 24940544
rbx 0x17c9008 24940552
rcx 0x7fffffffc670 140737488340592
rdx 0x17c9500 24941824
rsi 0x140 320
rdi 0x17c8ec0 24940224
rbp 0x8 0x8
rsp 0x10001000 0x10001000
r8 0x1721c40 24255552
r9 0x0 0
r10 0x140 320
r11 0x0 0
r12 0x178cf88 24694664
r13 0x1721c48 24255560
r14 0x8 8
r15 0x140 320
rip 0xbd4cfd 0xbd4cfd <dering_SSE2+205>
eflags 0x10293 [ CF AF SF IF RF ]
cs 0x33 51
ss 0x2b 43
ds 0x0 0
es 0x0 0
fs 0x0 0
gs 0x0 0
st0 -nan(0x1010101010101010) (raw 0xffff1010101010101010)
st1 -nan(0x010101010) (raw 0xffff0000000010101010)
st2 -nan(0x010101010) (raw 0xffff0000000010101010)
st3 -nan(0x010101010) (raw 0xffff0000000010101010)
st4 -nan(0x1010101010101010) (raw 0xffff1010101010101010)
st5 0 (raw 0x00000000000000000000)
st6 -nan(0x1000100010001000) (raw 0xffff1000100010001000)
st7 -nan(0x10001000100010) (raw 0xffff0010001000100010)
fctrl 0x37f 895
fstat 0x0 0
ftag 0xa6aa 42666
fiseg 0x0 0
fioff 0x0 0
foseg 0x0 0
fooff 0x0 0
fop 0x0 0
mxcsr 0x1fa0 [ PE IM DM ZM OM UM PM ]
ymm0 {v8_float = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v4_double = {0x0, 0x0, 0x0, 0x0}, v32_int8 = {0x10 <repeats 16 times>, 0x0 <repeats 16 times>}, v16_int16 = {0x1010, 0x1010, 0x1010, 0x1010, 0x1010, 0x1010, 0x1010, 0x1010, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v8_int32 = {0x10101010, 0x10101010, 0x10101010, 0x10101010, 0x0, 0x0, 0x0, 0x0}, v4_int64 = {0x1010101010101010, 0x1010101010101010, 0x0, 0x0}, v2_int128 = {0x10101010101010101010101010101010, 0x00000000000000000000000000000000}}
ymm1 {v8_float = {0x96970000, 0x96970000, 0x96970000, 0x96970000, 0x0, 0x0, 0x0, 0x0}, v4_double = {0x8000000000000000, 0x8000000000000000, 0x0, 0x0}, v32_int8 = {0xd2 <repeats 16 times>, 0x0 <repeats 16 times>}, v16_int16 = {0xd2d2, 0xd2d2, 0xd2d2, 0xd2d2, 0xd2d2, 0xd2d2, 0xd2d2, 0xd2d2, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v8_int32 = {0xd2d2d2d2, 0xd2d2d2d2, 0xd2d2d2d2, 0xd2d2d2d2, 0x0, 0x0, 0x0, 0x0}, v4_int64 = {0xd2d2d2d2d2d2d2d2, 0xd2d2d2d2d2d2d2d2, 0x0, 0x0}, v2_int128 = {0xd2d2d2d2d2d2d2d2d2d2d2d2d2d2d2d2, 0x00000000000000000000000000000000}}
ymm2 {v8_float = {0x96970000, 0x96970000, 0x96970000, 0x96970000, 0x0, 0x0, 0x0, 0x0}, v4_double = {0x8000000000000000, 0x8000000000000000, 0x0, 0x0}, v32_int8 = {0xd2 <repeats 16 times>, 0x0 <repeats 16 times>}, v16_int16 = {0xd2d2, 0xd2d2, 0xd2d2, 0xd2d2, 0xd2d2, 0xd2d2, 0xd2d2, 0xd2d2, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v8_int32 = {0xd2d2d2d2, 0xd2d2d2d2, 0xd2d2d2d2, 0xd2d2d2d2, 0x0, 0x0, 0x0, 0x0}, v4_int64 = {0xd2d2d2d2d2d2d2d2, 0xd2d2d2d2d2d2d2d2, 0x0, 0x0}, v2_int128 = {0xd2d2d2d2d2d2d2d2d2d2d2d2d2d2d2d2, 0x00000000000000000000000000000000}}
ymm3 {v8_float = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v4_double = {0x0, 0x0, 0x0, 0x0}, v32_int8 = {0x29 <repeats 16 times>, 0x0 <repeats 16 times>}, v16_int16 = {0x2929, 0x2929, 0x2929, 0x2929, 0x2929, 0x2929, 0x2929, 0x2929, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v8_int32 = {0x29292929, 0x29292929, 0x29292929, 0x29292929, 0x0, 0x0, 0x0, 0x0}, v4_int64 = {0x2929292929292929, 0x2929292929292929, 0x0, 0x0}, v2_int128 = {0x29292929292929292929292929292929, 0x00000000000000000000000000000000}}
ymm4 {v8_float = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v4_double = {0x0, 0x0, 0x0, 0x0}, v32_int8 = {0x29 <repeats 16 times>, 0x0 <repeats 16 times>}, v16_int16 = {0x2929, 0x2929, 0x2929, 0x2929, 0x2929, 0x2929, 0x2929, 0x2929, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v8_int32 = {0x29292929, 0x29292929, 0x29292929, 0x29292929, 0x0, 0x0, 0x0, 0x0}, v4_int64 = {0x2929292929292929, 0x2929292929292929, 0x0, 0x0}, v2_int128 = {0x29292929292929292929292929292929, 0x00000000000000000000000000000000}}
ymm5 {v8_float = {0x15151000, 0x15151000, 0x15151000, 0x15151000, 0x0, 0x0, 0x0, 0x0}, v4_double = {0x8000000000000000, 0x8000000000000000, 0x0, 0x0}, v32_int8 = {0x51 <repeats 16 times>, 0x0 <repeats 16 times>}, v16_int16 = {0x5151, 0x5151, 0x5151, 0x5151, 0x5151, 0x5151, 0x5151, 0x5151, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v8_int32 = {0x51515151, 0x51515151, 0x51515151, 0x51515151, 0x0, 0x0, 0x0, 0x0}, v4_int64 = {0x5151515151515151, 0x5151515151515151, 0x0, 0x0}, v2_int128 = {0x51515151515151515151515151515151, 0x00000000000000000000000000000000}}
ymm6 {v8_float = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v4_double = {0x0, 0x0, 0x0, 0x0}, v32_int8 = {0x0, 0x1, 0x40, 0x0, 0x0, 0x1, 0x40, 0x0, 0x0, 0x1, 0x40, 0x0, 0x0, 0x1, 0x40, 0x0 <repeats 17 times>}, v16_int16 = {0x100, 0x40, 0x100, 0x40, 0x100, 0x40, 0x100, 0x40, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v8_int32 = {0x400100, 0x400100, 0x400100, 0x400100, 0x0, 0x0, 0x0, 0x0}, v4_int64 = {0x40010000400100, 0x40010000400100, 0x0, 0x0}, v2_int128 = {0x00400100004001000040010000400100, 0x00000000000000000000000000000000}}
ymm7 {v8_float = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v4_double = {0x0, 0x0, 0x0, 0x0}, v32_int8 = {0x0, 0x80, 0x1, 0x80, 0x2, 0x80, 0x3, 0x80, 0x6, 0x80, 0x7, 0x80, 0x8, 0x80, 0x9, 0x80, 0x0 <repeats 16 times>}, v16_int16 = {0x8000, 0x8001, 0x8002, 0x8003, 0x8006, 0x8007, 0x8008, 0x8009, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v8_int32 = {0x80018000, 0x80038002, 0x80078006, 0x80098008, 0x0, 0x0, 0x0, 0x0}, v4_int64 = {0x8003800280018000, 0x8009800880078006, 0x0, 0x0}, v2_int128 = {0x80098008800780068003800280018000, 0x00000000000000000000000000000000}}
ymm8 {v8_float = {0x80000000, 0x0, 0x80000000, 0x0, 0x0, 0x0, 0x0, 0x0}, v4_double = {0x8000000000000000, 0x8000000000000000, 0x0, 0x0}, v32_int8 = {0xff, 0xec, 0xc8, 0xda, 0x0, 0x0, 0xff, 0xec, 0xff, 0xec, 0xc8, 0xda, 0x0, 0x0, 0xff, 0xec, 0x0 <repeats 16 times>}, v16_int16 = {0xecff, 0xdac8, 0x0, 0xecff, 0xecff, 0xdac8, 0x0, 0xecff, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v8_int32 = {0xdac8ecff, 0xecff0000, 0xdac8ecff, 0xecff0000, 0x0, 0x0, 0x0, 0x0}, v4_int64 = {0xecff0000dac8ecff, 0xecff0000dac8ecff, 0x0, 0x0}, v2_int128 = {0xecff0000dac8ecffecff0000dac8ecff, 0x00000000000000000000000000000000}}
ymm9 {v8_float = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v4_double = {0x0, 0x0, 0x0, 0x0}, v32_int8 = {0x38, 0x38, 0x0, 0x0, 0xc8, 0xda, 0x38, 0x38, 0x38, 0x38, 0x0, 0x0, 0xc8, 0xda, 0x38, 0x38, 0x0 <repeats 16 times>}, v16_int16 = {0x3838, 0x0, 0xdac8, 0x3838, 0x3838, 0x0, 0xdac8, 0x3838, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v8_int32 = {0x3838, 0x3838dac8, 0x3838, 0x3838dac8, 0x0, 0x0, 0x0, 0x0}, v4_int64 = {0x3838dac800003838, 0x3838dac800003838, 0x0, 0x0}, v2_int128 = {0x3838dac8000038383838dac800003838, 0x00000000000000000000000000000000}}
ymm10 {v8_float = {0xe63e4000, 0x0, 0xe63e4000, 0x0, 0x0, 0x0, 0x0, 0x0}, v4_double = {0x0, 0x0, 0x0, 0x0}, v32_int8 = {0x38, 0x38, 0xe3, 0xd0, 0x0, 0x0, 0x38, 0x38, 0x38, 0x38, 0xe3, 0xd0, 0x0, 0x0, 0x38, 0x38, 0x0 <repeats 16 times>}, v16_int16 = {0x3838, 0xd0e3, 0x0, 0x3838, 0x3838, 0xd0e3, 0x0, 0x3838, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v8_int32 = {0xd0e33838, 0x38380000, 0xd0e33838, 0x38380000, 0x0, 0x0, 0x0, 0x0}, v4_int64 = {0x38380000d0e33838, 0x38380000d0e33838, 0x0, 0x0}, v2_int128 = {0x38380000d0e3383838380000d0e33838, 0x00000000000000000000000000000000}}
ymm11 {v8_float = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v4_double = {0x8000000000000000, 0x8000000000000000, 0x0, 0x0}, v32_int8 = {0xe4, 0xf6, 0x0, 0x0, 0xe3, 0xd0, 0xe4, 0xf6, 0xe4, 0xf6, 0x0, 0x0, 0xe3, 0xd0, 0xe4, 0xf6, 0x0 <repeats 16 times>}, v16_int16 = {0xf6e4, 0x0, 0xd0e3, 0xf6e4, 0xf6e4, 0x0, 0xd0e3, 0xf6e4, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v8_int32 = {0xf6e4, 0xf6e4d0e3, 0xf6e4, 0xf6e4d0e3, 0x0, 0x0, 0x0, 0x0}, v4_int64 = {0xf6e4d0e30000f6e4, 0xf6e4d0e30000f6e4, 0x0, 0x0}, v2_int128 = {0xf6e4d0e30000f6e4f6e4d0e30000f6e4, 0x00000000000000000000000000000000}}
ymm12 {v8_float = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v4_double = {0x0, 0x0, 0x0, 0x0}, v32_int8 = {0x2, 0x80, 0x3, 0x80, 0x4, 0x80, 0x5, 0x80, 0x8, 0x80, 0x9, 0x80, 0xa, 0x80, 0xb, 0x80, 0x0 <repeats 16 times>}, v16_int16 = {0x8002, 0x8003, 0x8004, 0x8005, 0x8008, 0x8009, 0x800a, 0x800b, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v8_int32 = {0x80038002, 0x80058004, 0x80098008, 0x800b800a, 0x0, 0x0, 0x0, 0x0}, v4_int64 = {0x8005800480038002, 0x800b800a80098008, 0x0, 0x0}, v2_int128 = {0x800b800a800980088005800480038002, 0x00000000000000000000000000000000}}
ymm13 {v8_float = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v4_double = {0x0, 0x0, 0x0, 0x0}, v32_int8 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x42, 0xbc, 0x0 <repeats 24 times>}, v16_int16 = {0x0, 0x0, 0x0, 0xbc42, 0x0 <repeats 12 times>}, v8_int32 = {0x0, 0xbc420000, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v4_int64 = {0xbc42000000000000, 0x0, 0x0, 0x0}, v2_int128 = {0x0000000000000000bc42000000000000, 0x00000000000000000000000000000000}}
ymm14 {v8_float = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v4_double = {0x0, 0x0, 0x0, 0x0}, v32_int8 = {0xcb, 0x1a, 0xf2, 0x64, 0xae, 0xaa, 0x6c, 0xbc, 0x0 <repeats 24 times>}, v16_int16 = {0x1acb, 0x64f2, 0xaaae, 0xbc6c, 0x0 <repeats 12 times>}, v8_int32 = {0x64f21acb, 0xbc6caaae, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v4_int64 = {0xbc6caaae64f21acb, 0x0, 0x0, 0x0}, v2_int128 = {0x0000000000000000bc6caaae64f21acb, 0x00000000000000000000000000000000}}
ymm15 {v8_float = {0x0, 0x2, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v4_double = {0x3, 0x0, 0x0, 0x0}, v32_int8 = {0x0, 0xc6, 0xb8, 0xbe, 0xd3, 0xb9, 0xb, 0x40, 0x0 <repeats 24 times>}, v16_int16 = {0xc600, 0xbeb8, 0xb9d3, 0x400b, 0x0 <repeats 12 times>}, v8_int32 = {0xbeb8c600, 0x400bb9d3, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v4_int64 = {0x400bb9d3beb8c600, 0x0, 0x0, 0x0}, v2_int128 = {0x0000000000000000400bb9d3beb8c600, 0x00000000000000000000000000000000}}
$ valgrind ./ffmpeg_g -f lavfi -i testsrc -vf pp=dr -f null -
==15918== Memcheck, a memory error detector
==15918== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al.
==15918== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info
==15918== Command: ./ffmpeg_g -f lavfi -i testsrc -vf pp=dr -f null -
==15918==
ffmpeg version N-54036-g6c4516d Copyright (c) 2000-2013 the FFmpeg developers
built on Jun 16 2013 15:48:02 with gcc 4.7 (SUSE Linux)
configuration: --enable-gpl --disable-indev=jack --extra-cflags=-fstack-protector-all
libavutil 52. 35.101 / 52. 35.101
libavcodec 55. 16.100 / 55. 16.100
libavformat 55. 8.102 / 55. 8.102
libavdevice 55. 2.100 / 55. 2.100
libavfilter 3. 77.101 / 3. 77.101
libswscale 2. 3.100 / 2. 3.100
libswresample 0. 17.102 / 0. 17.102
libpostproc 52. 3.100 / 52. 3.100
Input #0, lavfi, from 'testsrc':
Duration: N/A, start: 0.000000, bitrate: N/A
Stream #0:0: Video: rawvideo (RGB[24] / 0x18424752), rgb24, 320x240 [SAR 1:1 DAR 4:3], 25 tbr, 25 tbn, 25 tbc
==15918== Invalid read of size 8
==15918== at 0x68DB59F: __GI___strncasecmp_l (in /lib64/libc-2.15.so)
==15918== by 0x688F8A5: ____strtod_l_internal (in /lib64/libc-2.15.so)
==15918== by 0xC5A83E: av_strtod (eval.c:96)
==15918== by 0xC5B0E4: parse_primary (eval.c:322)
==15918== by 0xC5BBA0: parse_factor (eval.c:481)
==15918== by 0xC5BDAB: parse_term (eval.c:530)
==15918== by 0xC5AE0E: parse_expr (eval.c:554)
==15918== by 0xC5BFED: av_expr_parse (eval.c:671)
==15918== by 0xC5C180: av_expr_parse_and_eval (eval.c:708)
==15918== by 0x4C3AAF: config_props (vf_scale.c:209)
==15918== by 0x48CC3E: avfilter_config_links (avfilter.c:242)
==15918== by 0x48CC21: avfilter_config_links (avfilter.c:231)
==15918== Address 0x746a780 is 0 bytes inside a block of size 3 alloc'd
==15918== at 0x4C290FE: memalign (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==15918== by 0x4C291A7: posix_memalign (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==15918== by 0xC6443F: av_malloc (mem.c:93)
==15918== by 0xC5BF3C: av_expr_parse (eval.c:648)
==15918== by 0xC5C180: av_expr_parse_and_eval (eval.c:708)
==15918== by 0x4C3AAF: config_props (vf_scale.c:209)
==15918== by 0x48CC3E: avfilter_config_links (avfilter.c:242)
==15918== by 0x48CC21: avfilter_config_links (avfilter.c:231)
==15918== by 0x490AEE: avfilter_graph_config (avfiltergraph.c:267)
==15918== by 0x47211E: configure_filtergraph (ffmpeg_filter.c:862)
==15918== by 0x47BA00: transcode_init (ffmpeg.c:2283)
==15918== by 0x466D3A: main (ffmpeg.c:3176)
==15918==
==15918== Invalid read of size 8
==15918== at 0x68DB5A7: __GI___strncasecmp_l (in /lib64/libc-2.15.so)
==15918== by 0x688F8A5: ____strtod_l_internal (in /lib64/libc-2.15.so)
==15918== by 0xC5A83E: av_strtod (eval.c:96)
==15918== by 0xC5B0E4: parse_primary (eval.c:322)
==15918== by 0xC5BBA0: parse_factor (eval.c:481)
==15918== by 0xC5BDAB: parse_term (eval.c:530)
==15918== by 0xC5AE0E: parse_expr (eval.c:554)
==15918== by 0xC5BFED: av_expr_parse (eval.c:671)
==15918== by 0xC5C180: av_expr_parse_and_eval (eval.c:708)
==15918== by 0x4C3AAF: config_props (vf_scale.c:209)
==15918== by 0x48CC3E: avfilter_config_links (avfilter.c:242)
==15918== by 0x48CC21: avfilter_config_links (avfilter.c:231)
==15918== Address 0x746a788 is 5 bytes after a block of size 3 alloc'd
==15918== at 0x4C290FE: memalign (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==15918== by 0x4C291A7: posix_memalign (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==15918== by 0xC6443F: av_malloc (mem.c:93)
==15918== by 0xC5BF3C: av_expr_parse (eval.c:648)
==15918== by 0xC5C180: av_expr_parse_and_eval (eval.c:708)
==15918== by 0x4C3AAF: config_props (vf_scale.c:209)
==15918== by 0x48CC3E: avfilter_config_links (avfilter.c:242)
==15918== by 0x48CC21: avfilter_config_links (avfilter.c:231)
==15918== by 0x490AEE: avfilter_graph_config (avfiltergraph.c:267)
==15918== by 0x47211E: configure_filtergraph (ffmpeg_filter.c:862)
==15918== by 0x47BA00: transcode_init (ffmpeg.c:2283)
==15918== by 0x466D3A: main (ffmpeg.c:3176)
==15918==
Output #0, null, to 'pipe:':
Metadata:
encoder : Lavf55.8.102
Stream #0:0: Video: rawvideo (444P / 0x50343434), yuv444p, 320x240 [SAR 1:1 DAR 4:3], q=2-31, 200 kb/s, 90k tbn, 25 tbc
Stream mapping:
Stream #0:0 -> #0:0 (rawvideo -> rawvideo)
Press [q] to stop, [?] for help
==15918== Warning: client switching stacks? SP change: 0x10001000 --> 0x0
==15918== to suppress, use: --max-stackframe=268439552 or greater
==15918==
==15918== Process terminating with default action of signal 11 (SIGSEGV)
==15918== Access not within mapped region at address 0x29
==15918== at 0xBD5291: dering_SSE2 (postprocess_template.c:1437)
==15918== If you believe this happened as a result of a stack
==15918== overflow in your program's main thread (unlikely but
==15918== possible), you can try to increase the size of the
==15918== main thread stack using the --main-stacksize= flag.
==15918== The main thread stack size used in this run was 8388608.
==15918==
==15918== HEAP SUMMARY:
==15918== in use at exit: 1,340,134 bytes in 237 blocks
==15918== total heap usage: 3,219 allocs, 2,982 frees, 3,375,434 bytes allocated
==15918==
==15918== LEAK SUMMARY:
==15918== definitely lost: 0 bytes in 0 blocks
==15918== indirectly lost: 0 bytes in 0 blocks
==15918== possibly lost: 4,896 bytes in 18 blocks
==15918== still reachable: 1,335,238 bytes in 219 blocks
==15918== suppressed: 0 bytes in 0 blocks
==15918== Rerun with --leak-check=full to see details of leaked memory
==15918==
==15918== For counts of detected and suppressed errors, rerun with: -v
==15918== ERROR SUMMARY: 6 errors from 2 contexts (suppressed: 2 from 2)
Killed
Change History (3)
comment:1 by , 11 years ago
comment:2 by , 11 years ago
| Resolution: | → fixed |
|---|---|
| Status: | new → closed |
Fixed / worked-around by Michael.
comment:3 by , 11 years ago
| Keywords: | SIGSEGV gcc added |
|---|
Note:
See TracTickets
for help on using tickets.
The reason is apparently that gcc miscompiles dering_SSE2() and dering_MMX2() in libpostproc/postproc_template.c:
1182 "movq %%mm6, %%mm0 \n\t" // max 1183 "psubb %%mm7, %%mm6 \n\t" // max - min 1184 "push %4 \n\t" 1184 "movd %%mm6, %k4 \n\t" 1185 "cmpb "MANGLE(deringThreshold)", %b4 \n\t" 1186 "pop %4 \n\t" ... : : "r" (src), "r" ((x86_reg)stride), "m" (c->pQPb), "m"(c->pQPb2), "q"(tmp) : "%"REG_a, "%"REG_d