h264: crash with forced codec and -max_alloc

[ami_stuff]

​http://www.datafilehost.com/d/e6b9258d

(gdb) r -max_alloc 10000000 -vcodec h264 -i dvvideo.avi -an -f null -
Starting program: D:\MinGW\msys\1.0\ffmpeg\ffmpeg_g.exe -max_alloc 10000000 -vco
dec h264 -i dvvideo.avi -an -f null -
[New Thread 3256.0xa20]
ffmpeg version 2.3.git Copyright (c) 2000-2014 the FFmpeg developers
  built on Aug 16 2014 20:43:17 with gcc 4.6.2 (GCC)
  configuration: --disable-yasm --disable-pthreads --disable-ffprobe --enable-gp
l
  libavutil      54.  4.100 / 54.  4.100
  libavcodec     56.  0.101 / 56.  0.101
  libavformat    56.  1.100 / 56.  1.100
  libavdevice    56.  0.100 / 56.  0.100
  libavfilter     5.  0.100 /  5.  0.100
  libswscale      3.  0.100 /  3.  0.100
  libswresample   1.  0.100 /  1.  0.100
  libpostproc    53.  0.100 / 53.  0.100
[h264 @ 059e5100] Warning: not compiled with thread support, using thread emulat
ion
[h264 @ 059e5100] no frame!
[pcm_s16le @ 05a1b560] Warning: not compiled with thread support, using thread e
mulation
[h264 @ 059e5100] no frame!
    Last message repeated 5 times
[h264 @ 059e5100] A non-intra slice in an IDR NAL unit.
[h264 @ 059e5100] decode_slice_header error
[h264 @ 059e5100] A non-intra slice in an IDR NAL unit.
[h264 @ 059e5100] decode_slice_header error
[h264 @ 059e5100] sps_id 32 out of range
    Last message repeated 1 times
[h264 @ 059e5100] no frame!
[h264 @ 059e5100] A non-intra slice in an IDR NAL unit.
[h264 @ 059e5100] decode_slice_header error
[h264 @ 059e5100] sps_id 32 out of range
    Last message repeated 1 times
[h264 @ 059e5100] illegal POC type 32
[h264 @ 059e5100] sps_id 32 out of range
[h264 @ 059e5100] no frame!
[h264 @ 059e5100] SEI type 127 size 1192 truncated at 5
[h264 @ 059e5100] illegal aspect ratio
[h264 @ 059e5100] too many reference frames 32
[...]
[h264 @ 061e40e0] reference count overflow
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] FMO not supported
[h264 @ 061e40e0] Reinit context to 32x1408, pix_fmt: yuvj420p
[h264 @ 061e40e0] reference picture missing during reorder
[h264 @ 061e40e0] Missing reference picture, default is 0
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] A non-intra slice in an IDR NAL unit.
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] illegal POC type 32
    Last message repeated 1 times
[h264 @ 061e40e0] Reinit context to 16x256, pix_fmt: yuvj420p
[h264 @ 061e40e0] Missing reference picture, default is 0
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] sps_id 4 out of range
[h264 @ 061e40e0] A non-intra slice in an IDR NAL unit.
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] FMO not supported
[h264 @ 061e40e0] A non-intra slice in an IDR NAL unit.
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] Missing reference picture, default is 0
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] top block unavailable for requested intra mode at 0 0
[h264 @ 061e40e0] error while decoding MB 0 0
[h264 @ 061e40e0] concealing 16 DC, 16 AC, 16 MV errors in I frame
Input stream #0:0 frame changed from size:32x16 fmt:yuvj420p to size:16x256 fmt:
yuvj420p
[swscaler @ 059d0060] deprecated pixel format used, make sure you did set range
correctly
[h264 @ 061e40e0] deblocking filter parameters 146 0 out of range
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] deblocking filter parameters 12 24 out of range
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] slice type 21 too large at 0 0
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] slice type 25 too large at 0 0
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] non-existing PPS 30 referenced
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] abs_diff_pic_num overflow
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] illegal aspect ratio
[h264 @ 061e40e0] sps_id 32 out of range
[h264 @ 061e40e0] A non-intra slice in an IDR NAL unit.
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] A non-intra slice in an IDR NAL unit.
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] slice type 13 too large at 0 5
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] sps_id 5 out of range
[h264 @ 061e40e0] Partitioned H.264 support is incomplete
[h264 @ 061e40e0] reference picture missing during reorder
[h264 @ 061e40e0] reference count overflow
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] A non-intra slice in an IDR NAL unit.
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] A non-intra slice in an IDR NAL unit.
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] slice type 32 too large at 0 0
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] FMO not supported
[h264 @ 061e40e0] A non-intra slice in an IDR NAL unit.
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] Missing reference picture, default is 0
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] non-existing PPS 29 referenced
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] slice type 14 too large at 0 0
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] reference overflow 0 > 15 or 45132 > 15
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] Missing reference picture, default is 0
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] slice type 25 too large at 0 5
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] non-existing PPS 30 referenced
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] illegal POC type 26
    Last message repeated 1 times
[h264 @ 061e40e0] A non-intra slice in an IDR NAL unit.
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] Reinit context to 32x1408, pix_fmt: yuvj420p
[h264 @ 061e40e0] deblocking_filter_idc 5 out of range
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] slice type 32 too large at 0 1
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] FMO not supported
[h264 @ 061e40e0] Reinit context to 16x256, pix_fmt: yuvj420p
[h264 @ 061e40e0] Missing reference picture, default is 0
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] slice type 17 too large at 0 0
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] Reinit context to 32x64, pix_fmt: yuvj420p
[h264 @ 061e40e0] first_mb_in_slice overflow
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] slice type 13 too large at 0 0
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] slice type 32 too large at 0 0
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] FMO not supported
    Last message repeated 1 times
[h264 @ 061e40e0] Reinit context to 16x256, pix_fmt: yuvj420p
[h264 @ 061e40e0] Missing reference picture, default is 0
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] Partitioned H.264 support is incomplete
[h264 @ 061e40e0] reference picture missing during reorder
    Last message repeated 1 times
[h264 @ 061e40e0] reference count overflow
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] Reinit context to 32x1408, pix_fmt: yuvj420p
[h264 @ 061e40e0] A non-intra slice in an IDR NAL unit.
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] FMO not supported
[h264 @ 061e40e0] Reinit context to 16x256, pix_fmt: yuvj420p
[h264 @ 061e40e0] long_term_pic_idx overflow
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] sps_id 32 out of range
    Last message repeated 1 times
[h264 @ 061e40e0] sps_id 29 out of range
[h264 @ 061e40e0] reference picture missing during reorder
[h264 @ 061e40e0] reference count overflow
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] Partitioned H.264 support is incomplete
[h264 @ 061e40e0] reference overflow (pps)
[h264 @ 061e40e0] A non-intra slice in an IDR NAL unit.
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] Partitioned H.264 support is incomplete
[h264 @ 061e40e0] A non-intra slice in an IDR NAL unit.
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] A non-intra slice in an IDR NAL unit.
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] Partitioned H.264 support is incomplete
[h264 @ 061e40e0] deblocking_filter_idc 4 out of range
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] reference picture missing during reorder
[h264 @ 061e40e0] reference count overflow
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] Missing reference picture, default is 0
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] A non-intra slice in an IDR NAL unit.
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] top block unavailable for requested intra mode at 0 1
[h264 @ 061e40e0] error while decoding MB 0 1
[h264 @ 061e40e0] Changing field mode (2 -> 3) between slices is not allowed
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] slice type 30 too large at 0 1
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] SEI type 132 size 1408 truncated at 111
[h264 @ 061e40e0] Partitioned H.264 support is incomplete
    Last message repeated 1 times
[h264 @ 061e40e0] Reinit context to 32x1408, pix_fmt: yuvj420p
[h264 @ 061e40e0] deblocking_filter_idc 32 out of range
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] SEI type 132 size 1408 truncated at 119
[h264 @ 061e40e0] Partitioned H.264 support is incomplete
[h264 @ 061e40e0] Reinit context to 560x416, pix_fmt: yuvj420p
[h264 @ 061e40e0] Missing reference picture, default is 0
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] non-existing PPS 22 referenced
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] FMO not supported
[h264 @ 061e40e0] Reinit context to 16x256, pix_fmt: yuvj420p
[h264 @ 061e40e0] first_mb_in_slice overflow
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] slice type 20 too large at 2 0
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] log2_max_poc_lsb (471) is out of range
[h264 @ 061e40e0] log2_max_poc_lsb (-1) is out of range
[h264 @ 061e40e0] FMO not supported
[h264 @ 061e40e0] reference overflow (pps)
[h264 @ 061e40e0] Partitioned H.264 support is incomplete
[h264 @ 061e40e0] A non-intra slice in an IDR NAL unit.
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] A non-intra slice in an IDR NAL unit.
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] FMO not supported
[h264 @ 061e40e0] first_mb_in_slice overflow
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] illegal modification_of_pic_nums_idc 21
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] A non-intra slice in an IDR NAL unit.
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] slice type 32 too large at 0 2
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] reference overflow (pps)
[h264 @ 061e40e0] FMO not supported
    Last message repeated 1 times
[h264 @ 061e40e0] reference overflow (pps)
[h264 @ 061e40e0] Missing reference picture, default is 0
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] Partitioned H.264 support is incomplete
[h264 @ 061e40e0] slice type 10 too large at 0 0
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] Partitioned H.264 support is incomplete
[h264 @ 061e40e0] A non-intra slice in an IDR NAL unit.
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] Missing reference picture, default is 0
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] QP 4294967217 out of range
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] reference overflow 246 > 15 or 0 > 15
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] reference overflow 24647 > 31 or 0 > 31
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] A non-intra slice in an IDR NAL unit.
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] Missing reference picture, default is 0
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] Missing reference picture, default is 0
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] Partitioned H.264 support is incomplete
[h264 @ 061e40e0] Missing reference picture, default is 0
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] A non-intra slice in an IDR NAL unit.
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] QP 3109 out of range
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] slice type 32 too large at 0 0
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] cabac_init_idc 32 overflow
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] reference picture missing during reorder
[h264 @ 061e40e0] reference count overflow
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] FMO not supported
[h264 @ 061e40e0] reference overflow (pps)
[h264 @ 061e40e0] A non-intra slice in an IDR NAL unit.
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] FMO not supported
[h264 @ 061e40e0] sps_id 9 out of range
[h264 @ 061e40e0] A non-intra slice in an IDR NAL unit.
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] slice type 13 too large at 0 1
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] Partitioned H.264 support is incomplete
[h264 @ 061e40e0] A non-intra slice in an IDR NAL unit.
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] non-existing PPS 14 referenced
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] sps_id 3 out of range
[h264 @ 061e40e0] first_mb_in_slice overflow
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] A non-intra slice in an IDR NAL unit.
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] Missing reference picture, default is 0
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] Reinit context to 32x64, pix_fmt: yuvj420p
[h264 @ 061e40e0] Missing reference picture, default is 2147483647
    Last message repeated 3 times
[h264 @ 061e40e0] deblocking_filter_idc 6 out of range
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] Partitioned H.264 support is incomplete
[h264 @ 061e40e0] A non-intra slice in an IDR NAL unit.
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] non-existing PPS 21 referenced
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] Reinit context to 16x256, pix_fmt: yuvj420p
[h264 @ 061e40e0] QP 3109 out of range
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] FMO not supported
[h264 @ 061e40e0] Reinit context to 32x64, pix_fmt: yuvj420p
[h264 @ 061e40e0] first_mb_in_slice overflow
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] This stream was generated by a broken encoder, invalid 8x8 inf
erence
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] FMO not supported
[h264 @ 061e40e0] slice type 19 too large at 0 1
[h264 @ 061e40e0] decode_slice_header error
[h264 @ 061e40e0] Partitioned H.264 support is incomplete
[h264 @ 061e40e0] Reinit context to 131056x2016, pix_fmt: yuvj420p
[h264 @ 061e40e0] Cannot allocate memory.
[h264 @ 061e40e0] Could not allocate memory
[h264 @ 061e40e0] h264_slice_header_init() failed
[h264 @ 061e40e0] decode_slice_header error

Program received signal SIGSEGV, Segmentation fault.
0x008dad7b in release_unused_pictures (remove_current=<optimized out>,
    h=<optimized out>) at libavcodec/h264_slice.c:193
193             if (h->DPB[i].f.buf[0] && !h->DPB[i].reference &&
(gdb) bt
#0  0x008dad7b in release_unused_pictures (remove_current=<optimized out>,
    h=<optimized out>) at libavcodec/h264_slice.c:193
#1  h264_frame_start (h=0x5b60040) at libavcodec/h264_slice.c:725
#2  0x008dd807 in ff_h264_decode_slice_header (h=0x5b60040, h0=0x5b60040)
    at libavcodec/h264_slice.c:1678
#3  0x005f7f9c in decode_nal_units (h=0x5b60040, buf=<optimized out>,
    buf_size=144000, parse_extradata=0) at libavcodec/h264.c:1588
#4  0x005f9728 in h264_decode_frame (avctx=0x61e40e0, data=0x5a5b000,
    got_frame=0x22facc, avpkt=0x22f798) at libavcodec/h264.c:1904
#5  0x005eb0f6 in avcodec_decode_video2 (avctx=0x61e40e0, picture=0x5a5b000,
    got_picture_ptr=0x22facc, avpkt=0x22f8c8) at libavcodec/utils.c:2264
#6  0x0041afc9 in decode_video (got_output=<optimized out>,
    pkt=<optimized out>, ist=<optimized out>) at ffmpeg.c:1888
#7  process_input_packet (ist=0x5defd40, pkt=0x22fb90) at ffmpeg.c:2122
#8  0x0041dd16 in process_input (file_index=2) at ffmpeg.c:3529
#9  0x00c8cf0c in transcode_step () at ffmpeg.c:3623
#10 transcode () at ffmpeg.c:3675
#11 main (argc=11, argv=<optimized out>) at ffmpeg.c:3851
(gdb)

[Carl Eugen Hoyos]

Patch sent.

[Carl Eugen Hoyos]

Fixed by Michael in 1fa35e43