Opened 10 years ago

Closed 10 years ago

#4151 closed defect (fixed)

Crash the ffmpeg then convert YV12 (yuv420p) to NV12 if width is less than 32

Reported by: v0lt Owned by:
Priority: important Component: swscale
Version: git-master Keywords: crash SIGSEGV
Cc: Blocked By:
Blocking: Reproduced by developer: yes
Analyzed by developer: no

Description

Summary of the bug:
I have same samples with lagarith YV12 (yuv420p). When the width is less than 32 fmppeg crash when converting to NV12.

How to reproduce:

% ffmpeg -i bw16x16_yv12.avi -vcodec rawvideo -pix_fmt nv12 bw16x16_nv12.avi
ffmpeg version N-68141-g4280150
built on Dec  1 2014 22:02:05 with gcc 4.9.2 (GCC)

Attachments (1)

bw16x16_yv12.avi​ (35.9 KB ) - added by v0lt 10 years ago.

Download all attachments as: .zip

Change History (4)

by v0lt, 10 years ago

Attachment: bw16x16_yv12.avi​ added

comment:1 by Carl Eugen Hoyos, 10 years ago

Keywords: crash SIGSEGV added
Priority: normal → important
Reproduced by developer: set
Status: new → open

For future tickets: Please always provide your failing command line together with the complete, uncut console output.

(gdb) r -f lavfi -i color=s=16x16 -pix_fmt nv12 -f null -
Starting program: ffmpeg_g -f lavfi -i color=s=16x16 -pix_fmt nv12 -f null -
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
ffmpeg version N-68146-gd771696 Copyright (c) 2000-2014 the FFmpeg developers
  built on Dec  2 2014 16:49:09 with gcc 4.7 (SUSE Linux)
  configuration: --enable-gpl
  libavutil      54. 15.100 / 54. 15.100
  libavcodec     56. 13.100 / 56. 13.100
  libavformat    56. 15.101 / 56. 15.101
  libavdevice    56.  3.100 / 56.  3.100
  libavfilter     5.  2.103 /  5.  2.103
  libswscale      3.  1.101 /  3.  1.101
  libswresample   1.  1.100 /  1.  1.100
  libpostproc    53.  3.100 / 53.  3.100
[New Thread 0x7ffff14f0700 (LWP 18162)]
[New Thread 0x7ffff0cef700 (LWP 18163)]
[New Thread 0x7ffff04ee700 (LWP 18164)]
[New Thread 0x7fffefced700 (LWP 18165)]
[New Thread 0x7fffef4ec700 (LWP 18166)]
[New Thread 0x7fffeeceb700 (LWP 18167)]
[New Thread 0x7fffee4ea700 (LWP 18168)]
[New Thread 0x7fffedce9700 (LWP 18169)]
[New Thread 0x7fffed4e8700 (LWP 18170)]
Input #0, lavfi, from 'color=s=16x16':
  Duration: N/A, start: 0.000000, bitrate: N/A
    Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 16x16 [SAR 1:1 DAR 1:1], 25 tbr, 25 tbn, 25 tbc
[New Thread 0x7fffecce7700 (LWP 18171)]
[New Thread 0x7fffec4e6700 (LWP 18172)]
[New Thread 0x7fffebce5700 (LWP 18173)]
[New Thread 0x7fffeb4e4700 (LWP 18174)]
[New Thread 0x7fffeace3700 (LWP 18175)]
[New Thread 0x7fffea4e2700 (LWP 18176)]
[New Thread 0x7fffe9ce1700 (LWP 18177)]
[New Thread 0x7fffe94e0700 (LWP 18178)]
[New Thread 0x7fffe8cdf700 (LWP 18179)]
Output #0, null, to 'pipe:':
  Metadata:
    encoder         : Lavf56.15.101
    Stream #0:0: Video: rawvideo (NV12 / 0x3231564E), nv12, 16x16 [SAR 1:1 DAR 1:1], q=2-31, 200 kb/s, 25 fps, 25 tbn, 25 tbc
    Metadata:
      encoder         : Lavc56.13.100 rawvideo
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> rawvideo (native))
Press [q] to stop, [?] for help

Program received signal SIGSEGV, Segmentation fault.
0x0000000000e477af in interleaveBytes_sse2 (
    src1=0x1a73980 "\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200",
    src2=0x1a739c0 "\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200",
    dest=0x1a68660 "\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200", width=8, height=8,
    src1Stride=8, src2Stride=8, dstStride=32) at libswscale/x86/rgb2rgb_template.c:1891
1891            __asm__(
(gdb) disass $pc-32,$pc+32
Dump of assembler code from 0xe4778f to 0xe477cf:
   0x0000000000e4778f <interleaveBytes_sse2+79>:        mov    %rax,-0x8(%rsp)
   0x0000000000e47794 <interleaveBytes_sse2+84>:        nopl   0x0(%rax)
   0x0000000000e47798 <interleaveBytes_sse2+88>:        xor    %rax,%rax
   0x0000000000e4779b <interleaveBytes_sse2+91>:        prefetchnta 0x40(%rdi,%rax,1)
   0x0000000000e477a0 <interleaveBytes_sse2+96>:        prefetchnta 0x40(%rsi,%rax,1)
   0x0000000000e477a5 <interleaveBytes_sse2+101>:       movdqa (%rdi,%rax,1),%xmm0
   0x0000000000e477aa <interleaveBytes_sse2+106>:       movdqa (%rdi,%rax,1),%xmm1
=> 0x0000000000e477af <interleaveBytes_sse2+111>:       movdqa (%rsi,%rax,1),%xmm2
   0x0000000000e477b4 <interleaveBytes_sse2+116>:       punpcklbw %xmm2,%xmm0
   0x0000000000e477b8 <interleaveBytes_sse2+120>:       punpckhbw %xmm2,%xmm1
   0x0000000000e477bc <interleaveBytes_sse2+124>:       movntdq %xmm0,(%rdx,%rax,2)
   0x0000000000e477c1 <interleaveBytes_sse2+129>:       movntdq %xmm1,0x10(%rdx,%rax,2)
   0x0000000000e477c7 <interleaveBytes_sse2+135>:       add    $0x10,%rax
   0x0000000000e477cb <interleaveBytes_sse2+139>:       cmp    %r13,%rax
   0x0000000000e477ce <interleaveBytes_sse2+142>:       jb     0xe4779b <interleaveBytes_sse2+91>
End of assembler dump.
(gdb) info all-register
rax            0x4640   17984
rbx            0x0      0
rcx            0x8      8
rdx            0x1a68660        27690592
rsi            0x1a739c0        27736512
rdi            0x1a73980        27736448
rbp            0x0      0x0
rsp            0x7fffffffd028   0x7fffffffd028
r8             0x8      8
r9             0x8      8
r10            0x0      0
r11            0x8      8
r12            0x0      0
r13            0xfffffffffffffff9       -7
r14            0x0      0
r15            0x10     16
rip            0xe477af 0xe477af <interleaveBytes_sse2+111>
eflags         0x10217  [ CF PF AF IF RF ]
cs             0x33     51
ss             0x2b     43
ds             0x0      0
es             0x0      0
fs             0x0      0
gs             0x0      0
st0            0        (raw 0x00000000000000000000)
st1            0        (raw 0x00000000000000000000)
st2            0        (raw 0x00000000000000000000)
st3            0        (raw 0x00000000000000000000)
st4            0        (raw 0x00000000000000000000)
st5            0        (raw 0x00000000000000000000)
st6            0        (raw 0x00000000000000000000)
st7            0        (raw 0x00000000000000000000)
fctrl          0x37f    895
fstat          0x0      0
ftag           0xffff   65535
fiseg          0x0      0
fioff          0x0      0
foseg          0x0      0
fooff          0x0      0
fop            0x0      0
mxcsr          0x1fa8   [ OE PE IM DM ZM OM UM PM ]
ymm0           {v8_float = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v4_double = {0x0, 0x0,
    0x0, 0x0}, v32_int8 = {0x0 <repeats 32 times>}, v16_int16 = {0x0 <repeats 16 times>},
  v8_int32 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v4_int64 = {0x0, 0x0, 0x0, 0x0},
  v2_int128 = {0x00000000000000000000000000000000, 0x00000000000000000000000000000000}}
ymm1           {v8_float = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v4_double = {0x0, 0x0,
    0x0, 0x0}, v32_int8 = {0x0 <repeats 32 times>}, v16_int16 = {0x0 <repeats 16 times>},
  v8_int32 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v4_int64 = {0x0, 0x0, 0x0, 0x0},
  v2_int128 = {0x00000000000000000000000000000000, 0x00000000000000000000000000000000}}
ymm2           {v8_float = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v4_double = {0x0, 0x0,
    0x0, 0x0}, v32_int8 = {0x0 <repeats 32 times>}, v16_int16 = {0x0 <repeats 16 times>},
  v8_int32 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v4_int64 = {0x0, 0x0, 0x0, 0x0},
  v2_int128 = {0x00000000000000000000000000000000, 0x00000000000000000000000000000000}}
ymm3           {v8_float = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v4_double = {0x0, 0x0,
    0x0, 0x0}, v32_int8 = {0x0 <repeats 32 times>}, v16_int16 = {0x0 <repeats 16 times>},
  v8_int32 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v4_int64 = {0x0, 0x0, 0x0, 0x0},
  v2_int128 = {0x00000000000000000000000000000000, 0x00000000000000000000000000000000}}
ymm4           {v8_float = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v4_double = {0x0,
    0x8000000000000000, 0x0, 0x0}, v32_int8 = {0x73, 0x6d, 0x70, 0x74, 0x65, 0x31, 0x37,
    0x30, 0x6d, 0x0, 0x49, 0x6e, 0x76, 0x61, 0x6c, 0x69, 0x0 <repeats 16 times>},
  v16_int16 = {0x6d73, 0x7470, 0x3165, 0x3037, 0x6d, 0x6e49, 0x6176, 0x696c, 0x0, 0x0, 0x0,
    0x0, 0x0, 0x0, 0x0, 0x0}, v8_int32 = {0x74706d73, 0x30373165, 0x6e49006d, 0x696c6176,
    0x0, 0x0, 0x0, 0x0}, v4_int64 = {0x3037316574706d73, 0x696c61766e49006d, 0x0, 0x0},
  v2_int128 = {0x696c61766e49006d3037316574706d73, 0x00000000000000000000000000000000}}
ymm5           {v8_float = {0x0, 0x1, 0x3, 0x3, 0x0, 0x0, 0x0, 0x0}, v4_double = {0x1, 0x20,
    0x0, 0x0}, v32_int8 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xf0, 0x3f, 0x40, 0x40, 0x40, 0x40,
    0x40, 0x40, 0x40, 0x40, 0x0 <repeats 16 times>}, v16_int16 = {0x0, 0x0, 0x0, 0x3ff0,
    0x4040, 0x4040, 0x4040, 0x4040, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v8_int32 = {
    0x0, 0x3ff00000, 0x40404040, 0x40404040, 0x0, 0x0, 0x0, 0x0}, v4_int64 = {
    0x3ff0000000000000, 0x4040404040404040, 0x0, 0x0}, v2_int128 = {
    0x40404040404040403ff0000000000000, 0x00000000000000000000000000000000}}
ymm6           {v8_float = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v4_double = {0x0,
    0x8000000000000000, 0x0, 0x0}, v32_int8 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x65,
    0x66, 0x63, 0x6f, 0x75, 0x6e, 0x74, 0x65, 0x0 <repeats 16 times>}, v16_int16 = {0x0,
    0x0, 0x0, 0x0, 0x6665, 0x6f63, 0x6e75, 0x6574, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0},
  v8_int32 = {0x0, 0x0, 0x6f636665, 0x65746e75, 0x0, 0x0, 0x0, 0x0}, v4_int64 = {0x0,
    0x65746e756f636665, 0x0, 0x0}, v2_int128 = {0x65746e756f6366650000000000000000,
    0x00000000000000000000000000000000}}
ymm7           {v8_float = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v4_double = {0x0, 0x0,
    0x0, 0x0}, v32_int8 = {0x20 <repeats 16 times>, 0x0 <repeats 16 times>}, v16_int16 = {
    0x2020, 0x2020, 0x2020, 0x2020, 0x2020, 0x2020, 0x2020, 0x2020, 0x0, 0x0, 0x0, 0x0, 0x0,
    0x0, 0x0, 0x0}, v8_int32 = {0x20202020, 0x20202020, 0x20202020, 0x20202020, 0x0, 0x0,
    0x0, 0x0}, v4_int64 = {0x2020202020202020, 0x2020202020202020, 0x0, 0x0}, v2_int128 = {
    0x20202020202020202020202020202020, 0x00000000000000000000000000000000}}
ymm8           {v8_float = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v4_double = {0x0, 0x0,
    0x0, 0x0}, v32_int8 = {0x0 <repeats 32 times>}, v16_int16 = {0x0 <repeats 16 times>},
  v8_int32 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v4_int64 = {0x0, 0x0, 0x0, 0x0},
  v2_int128 = {0x00000000000000000000000000000000, 0x00000000000000000000000000000000}}
ymm9           {v8_float = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v4_double = {
    0x8000000000000000, 0x8000000000000000, 0x0, 0x0}, v32_int8 = {0x0, 0x0,
    0xff <repeats 14 times>, 0x0 <repeats 16 times>}, v16_int16 = {0x0, 0xffff, 0xffff,
    0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0},
  v8_int32 = {0xffff0000, 0xffffffff, 0xffffffff, 0xffffffff, 0x0, 0x0, 0x0, 0x0},
  v4_int64 = {0xffffffffffff0000, 0xffffffffffffffff, 0x0, 0x0}, v2_int128 = {
    0xffffffffffffffffffffffffffff0000, 0x00000000000000000000000000000000}}
ymm10          {v8_float = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v4_double = {0x0, 0x0,
    0x0, 0x0}, v32_int8 = {0x0 <repeats 32 times>}, v16_int16 = {0x0 <repeats 16 times>},
  v8_int32 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v4_int64 = {0x0, 0x0, 0x0, 0x0},
  v2_int128 = {0x00000000000000000000000000000000, 0x00000000000000000000000000000000}}
ymm11          {v8_float = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v4_double = {0x0, 0x0,
    0x0, 0x0}, v32_int8 = {0x0, 0x0, 0x0, 0xff, 0x0, 0x0, 0x0, 0x0, 0x0, 0xff, 0x0, 0x0,
    0x0, 0xff, 0xff, 0x0 <repeats 17 times>}, v16_int16 = {0x0, 0xff00, 0x0, 0x0, 0xff00,
    0x0, 0xff00, 0xff, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v8_int32 = {0xff000000, 0x0,
    0xff00, 0xffff00, 0x0, 0x0, 0x0, 0x0}, v4_int64 = {0xff000000, 0xffff000000ff00, 0x0,
    0x0}, v2_int128 = {0x00ffff000000ff0000000000ff000000,
    0x00000000000000000000000000000000}}
ymm12          {v8_float = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v4_double = {0x0, 0x0,
    0x0, 0x0}, v32_int8 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xc4, 0x3c,
    0x0 <repeats 24 times>}, v16_int16 = {0x0, 0x0, 0x0, 0x3cc4, 0x0 <repeats 12 times>},
  v8_int32 = {0x0, 0x3cc40000, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v4_int64 = {
    0x3cc4000000000000, 0x0, 0x0, 0x0}, v2_int128 = {0x00000000000000003cc4000000000000,
    0x00000000000000000000000000000000}}
ymm13          {v8_float = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v4_double = {0x0, 0x0,
    0x0, 0x0}, v32_int8 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x80, 0x59, 0xbc,
    0x0 <repeats 24 times>}, v16_int16 = {0x0, 0x0, 0x8000, 0xbc59, 0x0 <repeats 12 times>},
  v8_int32 = {0x0, 0xbc598000, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v4_int64 = {
    0xbc59800000000000, 0x0, 0x0, 0x0}, v2_int128 = {0x0000000000000000bc59800000000000,
    0x00000000000000000000000000000000}}
ymm14          {v8_float = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v4_double = {0x0, 0x0,
    0x0, 0x0}, v32_int8 = {0x8e, 0x85, 0x83, 0xe8, 0xf0, 0x24, 0x53, 0x3c,
    0x0 <repeats 24 times>}, v16_int16 = {0x858e, 0xe883, 0x24f0, 0x3c53,
    0x0 <repeats 12 times>}, v8_int32 = {0xe883858e, 0x3c5324f0, 0x0, 0x0, 0x0, 0x0, 0x0,
    0x0}, v4_int64 = {0x3c5324f0e883858e, 0x0, 0x0, 0x0}, v2_int128 = {
    0x00000000000000003c5324f0e883858e, 0x00000000000000000000000000000000}}
ymm15          {v8_float = {0x0, 0x3, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v4_double = {0x2d, 0x0,
    0x0, 0x0}, v32_int8 = {0xc0, 0x9, 0xf2, 0x16, 0xb5, 0xdf, 0x46, 0x40,
    0x0 <repeats 24 times>}, v16_int16 = {0x9c0, 0x16f2, 0xdfb5, 0x4046,
    0x0 <repeats 12 times>}, v8_int32 = {0x16f209c0, 0x4046dfb5, 0x0, 0x0, 0x0, 0x0, 0x0,
    0x0}, v4_int64 = {0x4046dfb516f209c0, 0x0, 0x0, 0x0}, v2_int128 = {
    0x00000000000000004046dfb516f209c0, 0x00000000000000000000000000000000}}

comment:2 by Carl Eugen Hoyos, 10 years ago

Workaround is to compile with --disable-asm or --disable-mmx, --cpuflags 0 does not help.

comment:3 by Michael Niedermayer, 10 years ago

Resolution: → fixed
Status: open → closed

Fixed in 8524558858b7e14bc50afa10233e0194f591ab9d

Note: See TracTickets for help on using tickets.