jpeg2000: invalid write with lowres 3

[ami_stuff]

lowres doesn't work correctly with attached sample

for more samples (different color spaces) see ticket #2871

​http://www.datafilehost.com/d/8ae6bfef

knoppix@Microknoppix:/media/sdb1$ gdb ffmpeg-HEAD-edf6fb6/ffmpeg_g
GNU gdb (GDB) 7.4.1-debian
Copyright (C) 2012 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "i486-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /media/sdb1/ffmpeg-HEAD-edf6fb6/ffmpeg_g...done.
(gdb) r -vlowres 3 -i 444_layers1.avi -an -f null -
Starting program: /media/sdb1/ffmpeg-HEAD-edf6fb6/ffmpeg_g -vlowres 3 -i 444_layers1.avi -an -f null -
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/i386-linux-gnu/libthread_db.so.1".
ffmpeg version 2.0-edf6fb6 Copyright (c) 2000-2013 the FFmpeg developers
  built on Aug 24 2013 11:50:43 with gcc 4.7 (Debian 4.7.2-5)
  configuration: --disable-yasm --disable-ffserver --disable-ffprobe --enable-gpl
  libavutil      52. 42.100 / 52. 42.100
  libavcodec     55. 29.100 / 55. 29.100
  libavformat    55. 14.102 / 55. 14.102
  libavdevice    55.  3.100 / 55.  3.100
  libavfilter     3. 82.102 /  3. 82.102
  libswscale      2.  5.100 /  2.  5.100
  libswresample   0. 17.103 /  0. 17.103
  libpostproc    52.  3.100 / 52.  3.100

Program received signal SIGSEGV, Segmentation fault.
jpeg2000_decode_tile (s=s@entry=0x91097e0, tile=0x9140fc0, 
    picture=picture@entry=0x9140e00) at libavcodec/jpeg2000dec.c:1221
1221	                        decode_cblk(s, codsty, &t1, cblk,
(gdb) bt
#0  jpeg2000_decode_tile (s=s@entry=0x91097e0, tile=0x9140fc0, 
    picture=picture@entry=0x9140e00) at libavcodec/jpeg2000dec.c:1221
#1  0x0850d77e in jpeg2000_decode_frame (avctx=0x91066a0, data=0x9140e00, 
    got_frame=0xbffff030, avpkt=0xbfffefa8) at libavcodec/jpeg2000dec.c:1636
#2  0x08677b0e in avcodec_decode_video2 (avctx=0x91066a0, picture=0x9140e00, 
    got_picture_ptr=got_picture_ptr@entry=0xbffff030, 
    avpkt=avpkt@entry=0xbffff058) at libavcodec/utils.c:1982
#3  0x08233ef8 in try_decode_frame (st=st@entry=0x9106420, 
    avpkt=avpkt@entry=0x9140d80, options=0x9106ee0) at libavformat/utils.c:2463
#4  0x0823d681 in avformat_find_stream_info (ic=0x9105e40, options=0x9106ee0)
    at libavformat/utils.c:2908
#5  0x080a6325 in open_input_file (o=o@entry=0xbffff51c, 
    filename=<optimized out>) at ffmpeg_opt.c:809
#6  0x080a4b47 in open_files (inout=inout@entry=0x88d82db "input", 
    open_file=open_file@entry=0x80a5f40 <open_input_file>, 
    l=<error reading variable: Unhandled dwarf expression opcode 0xfa>, 
    l=<error reading variable: Unhandled dwarf expression opcode 0xfa>)
    at ffmpeg_opt.c:2494
#7  0x080acd59 in ffmpeg_parse_options (argc=argc@entry=9, 
    argv=argv@entry=0xbffff9a4) at ffmpeg_opt.c:2531
#8  0x080a224a in main (argc=9, argv=0xbffff9a4) at ffmpeg.c:3389
(gdb) 

knoppix@Microknoppix:/media/sdb1$ valgrind --leak-check=full ffmpeg-HEAD-edf6fb6/ffmpeg_g -vlowres 3 -i 444_layers1.avi -an -f null -
==10646== Memcheck, a memory error detector
==10646== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al.
==10646== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info
==10646== Command: ffmpeg-HEAD-edf6fb6/ffmpeg_g -vlowres 3 -i 444_layers1.avi -an -f null -
==10646== 
ffmpeg version 2.0-edf6fb6 Copyright (c) 2000-2013 the FFmpeg developers
  built on Aug 24 2013 11:50:43 with gcc 4.7 (Debian 4.7.2-5)
  configuration: --disable-yasm --disable-ffserver --disable-ffprobe --enable-gpl
  libavutil      52. 42.100 / 52. 42.100
  libavcodec     55. 29.100 / 55. 29.100
  libavformat    55. 14.102 / 55. 14.102
  libavdevice    55.  3.100 / 55.  3.100
  libavfilter     3. 82.102 /  3. 82.102
  libswscale      2.  5.100 /  2.  5.100
  libswresample   0. 17.103 /  0. 17.103
  libpostproc    52.  3.100 / 52.  3.100
==10646== Invalid write of size 4
==10646==    at 0x8509DBE: jpeg2000_decode_tile (jpeg2000dec.c:1098)
==10646==    by 0x850D77D: jpeg2000_decode_frame (jpeg2000dec.c:1636)
==10646==    by 0x8677B0D: avcodec_decode_video2 (utils.c:1982)
==10646==    by 0x8233EF7: try_decode_frame (utils.c:2463)
==10646==  Address 0x42d2cf0 is 0 bytes after a block of size 5,456 alloc'd
==10646==    at 0x40268A4: memalign (vg_replace_malloc.c:694)
==10646==    by 0x402695E: posix_memalign (vg_replace_malloc.c:835)
==10646==    by 0x886CFF7: av_malloc (mem.c:93)
==10646==    by 0x8508B9D: ff_jpeg2000_init_component (mem.h:98)
==10646==    by 0x850CB96: jpeg2000_decode_frame (jpeg2000dec.c:678)
==10646==    by 0x8677B0D: avcodec_decode_video2 (utils.c:1982)
==10646==    by 0x8233EF7: try_decode_frame (utils.c:2463)
==10646== 
==10646== Invalid read of size 2
==10646==    at 0x8509990: jpeg2000_decode_tile (jpeg2000dec.c:1221)
==10646==    by 0x850D77D: jpeg2000_decode_frame (jpeg2000dec.c:1636)
==10646==    by 0x8677B0D: avcodec_decode_video2 (utils.c:1982)
==10646==    by 0x8233EF7: try_decode_frame (utils.c:2463)
==10646==  Address 0x43e70984 is not stack'd, malloc'd or (recently) free'd
==10646== 
==10646== 
==10646== Process terminating with default action of signal 11 (SIGSEGV)
==10646==  Access not within mapped region at address 0x43E70984
==10646==    at 0x8509990: jpeg2000_decode_tile (jpeg2000dec.c:1221)
==10646==    by 0x850D77D: jpeg2000_decode_frame (jpeg2000dec.c:1636)
==10646==    by 0x8677B0D: avcodec_decode_video2 (utils.c:1982)
==10646==    by 0x8233EF7: try_decode_frame (utils.c:2463)
==10646==  If you believe this happened as a result of a stack
==10646==  overflow in your program's main thread (unlikely but
==10646==  possible), you can try to increase the size of the
==10646==  main thread stack using the --main-stacksize= flag.
==10646==  The main thread stack size used in this run was 8388608.
==10646== 
==10646== HEAP SUMMARY:
==10646==     in use at exit: 1,328,080 bytes in 164 blocks
==10646==   total heap usage: 280 allocs, 116 frees, 1,434,676 bytes allocated
==10646== 
==10646== 264 bytes in 1 blocks are definitely lost in loss record 25 of 50
==10646==    at 0x40268A4: memalign (vg_replace_malloc.c:694)
==10646==    by 0x402695E: posix_memalign (vg_replace_malloc.c:835)
==10646==    by 0x886D267: av_mallocz (mem.c:93)
==10646==    by 0x85077A6: ff_jpeg2000_tag_tree_init (mem.h:197)
==10646==    by 0x8508912: ff_jpeg2000_init_component (jpeg2000.c:416)
==10646==    by 0x850CB96: jpeg2000_decode_frame (jpeg2000dec.c:678)
==10646==    by 0x8677B0D: avcodec_decode_video2 (utils.c:1982)
==10646==    by 0x8233EF7: try_decode_frame (utils.c:2463)
==10646== 
==10646== 264 bytes in 1 blocks are definitely lost in loss record 26 of 50
==10646==    at 0x40268A4: memalign (vg_replace_malloc.c:694)
==10646==    by 0x402695E: posix_memalign (vg_replace_malloc.c:835)
==10646==    by 0x886D267: av_mallocz (mem.c:93)
==10646==    by 0x85077A6: ff_jpeg2000_tag_tree_init (mem.h:197)
==10646==    by 0x850892A: ff_jpeg2000_init_component (jpeg2000.c:422)
==10646==    by 0x850CB96: jpeg2000_decode_frame (jpeg2000dec.c:678)
==10646==    by 0x8677B0D: avcodec_decode_video2 (utils.c:1982)
==10646==    by 0x8233EF7: try_decode_frame (utils.c:2463)
==10646== 
==10646== LEAK SUMMARY:
==10646==    definitely lost: 528 bytes in 2 blocks
==10646==    indirectly lost: 0 bytes in 0 blocks
==10646==      possibly lost: 0 bytes in 0 blocks
==10646==    still reachable: 1,327,552 bytes in 162 blocks
==10646==         suppressed: 0 bytes in 0 blocks
==10646== Reachable blocks (those to which a pointer was found) are not shown.
==10646== To see them, rerun with: --leak-check=full --show-reachable=yes
==10646== 
==10646== For counts of detected and suppressed errors, rerun with: -v
==10646== ERROR SUMMARY: 308 errors from 4 contexts (suppressed: 59 from 6)
Segmentation fault

[Michael Niedermayer]

fixed in a1b9004b768bef606ee98d417bceb9392ceb788d