Several problems with prores decoders/encoders during compilation

[jamal]

Some problems i found with the different prores decoders and encoders.

First, the following configuration fails when linking the executables:

./configure --disable-everything --enable-decoder=prores

The error is as follows:

LD      ffmpeg_g.exe
libavcodec/libavcodec.a(proresdec2.o): In function `decode_init':
D:\msys\1.0\home\jamal\ffmpeg/libavcodec/proresdec2.c:73: undefined reference to `ff_proresdsp_init'
collect2: ld returned 1 exit status
make: *** [ffmpeg_g.exe] Error 1

The problem is that libavcodec/Makefile is not compiling proresdsp.c when this decoder is enabled alone.

OBJS-$(CONFIG_PRORES_DECODER)          += proresdec2.o

should be

OBJS-$(CONFIG_PRORES_DECODER)          += proresdec2.o proresdsp.o

---

Second, when for example configuring with

./configure --disable-decoders --enable-decoder=prores_lgpl

The compilation succeeds, but both ffmpeg and ffprobe crash when trying to read a file with a prores stream.
The problem is that proresdsp.c contains two functions used by the prores_lgpl decoder that only consider "CONFIG_PRORES_DECODER" but not "CONFIG_PRORES_LGPL_DECODER" during compilation.

In both cases

#if CONFIG_PRORES_DECODER

should be

#if CONFIG_PRORES_DECODER | CONFIG_PRORES_LGPL_DECODER

Since the other prores decoder also uses these functions.

---

Third, when compiling for example with

./configure --disable-encoders --enable-encoder=prores_kostya

The compilation succeeds as well, but ffmpeg crashes when trying to encode using said encoder.
The problem is once again in proresdsp.c where two functions needed by the prores_kostya encoder checks for CONFIG_PRORES_ENCODER when it should check for CONFIG_PRORES_KOSTYA_ENCODER instead.

In both cases

#if CONFIG_PRORES_ENCODER

should be

#if CONFIG_PRORES_KOSTYA_ENCODER

Since the other prores encoder doesn't use functions from proresdsp.c at all.

---

I presume most of these problem were introduced after the decoders/encoders got a name change in one of the merges from libav. The prores and prores_anatoliy encoders are exactly the same thing for example, yet for some reason are two separate encoders.

Once again, found this thanks to Måns' --enable-random feature.

[Carl Eugen Hoyos]

Should be fixed, thank you!