"Access violation reading location 0x00000000" using yadif/libavfilter on MSVC++

[Francesco Pretto]

Running the attached self compiling sample (copied from ​here, not really part of FFmpeg, it does use the libavfilter interface to do yadif deinterlacing on the FILENAME video) it's giving me "Unhandled exception at 0x6d783c8b in Test.exe: 0xC0000005: Access violation reading location 0x00000000." using ffmpeg *.dlls with MSVC++ (recent build ​here). It seems raicy, because it happens always without brekpoints, but sometimes it barely works stepping into it, but no threading involved (I guess). May it be alignment problems as some reported ​here?

[Francesco Pretto]

Forgot, the error triggers on avfilter_poll_frame() or avfilter_request_frame() functions.

[Carl Eugen Hoyos]

Is the problem also reproducible without MSVC++?
(And if not, why not?)

[Francesco Pretto]

Because of the raicy nature of the problem (it triggers almost always, but sometimes it works), and threading seems not involved (I just use blocking functions), I have suspects it's a stack alignment problem (citing ​here, "ffmpeg.exe (compiled with gcc) aligns main() to 16 bytes and the whole library maintains alignment. Code compiled with MSVC++ only aligns to 4 bytes. The library maintains the alignment, but doesn't realign to 16 bytes"). I'd like to test it with gcc but I don't have a build env ready to compile latest ffmpeg and all its dependencies. If you don't have any other hint I'll probably try to install a gentoo in a VM, compile ffmpeg and test there.

[Francesco Pretto]

Really seems alignment problems. Here is the disassembly were libavfilter crashes:

6D783C61 punpcklbw xmm0,xmm7
6D783C65 movq xmm1,mmword ptr [esi+eax]
6D783C6A punpcklbw xmm1,xmm7
6D783C6E movq xmm2,mmword ptr [ebx]
6D783C72 punpcklbw xmm2,xmm7
6D783C76 movq xmm3,mmword ptr [esi]
6D783C7A punpcklbw xmm3,xmm7
6D783C7E movdqa xmm4,xmm3
6D783C82 paddw xmm3,xmm2
6D783C86 psraw xmm3,1
6D783C8B movdqa xmmword ptr [esp+30h],xmm0 <----------------------------
6D783C91 movdqa xmmword ptr [esp+20h],xmm3
6D783C97 movdqa xmmword ptr [esp+10h],xmm1
6D783C9D psubw xmm2,xmm4
6D783CA1 pabsw xmm2,xmm2
6D783CA6 movq xmm3,mmword ptr [ebx+edx]
6D783CAB punpcklbw xmm3,xmm7
6D783CAF movq xmm4,mmword ptr [ebx+eax]
6D783CB4 punpcklbw xmm4,xmm7

Maybe libavfilter is not aligning aware like other parts of ffmpeg (looking around I saw ffmpeg must use its allocation functions internally)? Just decoding seems ok here. Sorry but can't debug more if not teached how to do so.

[Francesco Pretto]

Recompiling ffmpeg with --disable-sse and --disable-ssse3 fixes the problem here.

For some Euros, I may be interested in bounting the fix of this bug (ref[1])

Oh, I have found another possible bug (assertion failed). Just suppose I have the same filter chain: src_buffer and yadif. If I don't insert the first frame on src_buffer (so the buffer is empty) and I immediately poll the output link of yadif filter, I get:

Assertion failed!

Program: C:\Users\ceztko\Staging\target\Debug\Test.exe
File: libavfilter/vf_yadif.c, Line 277

Expression: yadif->next

...and the executable quits.

[1] ​http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/2010-January/082413.html

[Carl Eugen Hoyos]

Please open a new ticket for the assertion you discovered (the command line is most important).

Does the crash you see (with --enable-sse) happen with ffmpeg (the executable) or only with your MSVC application?

[Francesco Pretto]

Replying to cehoyos:

Please open a new ticket for the assertion you discovered
(the command line is most important).

It's not a problem with ffmpeg.exe, but with the api: maybe it's debatable but with src_buffer empty, polling yadif for frames should just return 0 IMO, not failing. Will create a new ticket later with explanation and references to this bug.

Does the crash you see (with --enable-sse) happen with ffmpeg (the executable)
or only with your MSVC application?

I wasn't able to reproduce with ffmpeg.exe (that's compiled with gcc). I think ffmpeg is doing the same thing with filter chains, tough.

[Francesco Pretto]

c++ sample, gcc compatible. gcc --> ok, MSVC --> crash

[Francesco Pretto]

Replying to cehoyos:

Does the crash you see (with --enable-sse) happen with
ffmpeg (the executable) or only with your MSVC application?

I verified the problem is compiler specific testing it in gcc.

The attacched (test.tar.bz2) c++ sample works in gcc with SSE extensions enabled. With MSVC++ --> crash. It just need a (possibly interlaced) video to be set in the
Test.cpp source.

Follows compile instructions:

Reflect include/ and /lib dirs to the correct headers and
shared objects locations of ffmpeg

g++ -c -o libav.o -Iinclude/ -DSTDC_CONSTANT_MACROS libav.cpp
g++ -c -o Test.o -Iinclude/ -DSTDC_CONSTANT_MACROS Test.cpp
g++ -o Test libav.o -Llib/ -llibavformat -llibavcodec -llibavdevice -llibavutil -llibavfilter -llibswscale Test.o

[Carl Eugen Hoyos]

So am I correct that this is not a FFmpeg problem or do I misunderstand?

[Francesco Pretto]

Replying to cehoyos:

So am I correct that this is not a FFmpeg problem or do I misunderstand?

I guess it's a memory alignment related problem as other fixed in
ffmpeg in the past (optimized SSE code is expecting to find data
aligned in the memory, but maybe yadif/libavfilter don't take
safeguards to enforce this), but can't confirm this because I don't
have the required assembly expertise. I realize that this problem is
hard to debug if the devs don't have the platform that triggers the
problem, but please don't mark this as "not our problem" until you are
really sure.

[Michael Niedermayer]

Should be fixed, please reopen if its not so. I could not test due to not having msvc nor windows

[Francesco Pretto]

Thanks! Can't test now but will do it as soon as possible.