YASM Generation error in dsputil_yasm.asm & workaround

[Gilles Sabourin]

Since August, 16th, i have problem to generate "libavcodec/x86/dsputil_yasm.asm". It gives the following errors :

YASM libavcodec/x86/dsputil_yasm.o
libavcodec/x86/dsputil_yasm.asm:1122: error: instruction expected after label
libavcodec/x86/dsputil_yasm.asm:1122: error: instruction expected after label
libavcodec/x86/dsputil_yasm.asm:1122: error: instruction expected after label
libavcodec/x86/dsputil_yasm.asm:1122: error: instruction expected after label
libavcodec/x86/dsputil_yasm.asm:1122: error: undefined symbol `CLIPD_MMX.loop' (first use)
libavcodec/x86/dsputil_yasm.asm:1122: error: (Each undefined symbol is reported only once.)
libavcodec/x86/dsputil_yasm.asm:1125: error: instruction expected after label
libavcodec/x86/dsputil_yasm.asm:1125: error: instruction expected after label
libavcodec/x86/dsputil_yasm.asm:1125: error: instruction expected after label
libavcodec/x86/dsputil_yasm.asm:1125: error: instruction expected after label
libavcodec/x86/dsputil_yasm.asm:1127: error: instruction expected after label
libavcodec/x86/dsputil_yasm.asm:1127: error: instruction expected after label
libavcodec/x86/dsputil_yasm.asm:1127: error: instruction expected after label
libavcodec/x86/dsputil_yasm.asm:1127: error: instruction expected after label
libavcodec/x86/dsputil_yasm.asm:1127: error: instruction expected after label
libavcodec/x86/dsputil_yasm.asm:1127: error: instruction expected after label
libavcodec/x86/dsputil_yasm.asm:1127: error: instruction expected after label
libavcodec/x86/dsputil_yasm.asm:1127: error: instruction expected after label
libavcodec/x86/dsputil_yasm.asm:1127: error: undefined symbol `CLIPD_SSE2.loop' (first use)
libavcodec/x86/dsputil_yasm.asm:1130: error: instruction expected after label
libavcodec/x86/dsputil_yasm.asm:1130: error: instruction expected after label
libavcodec/x86/dsputil_yasm.asm:1130: error: instruction expected after label
libavcodec/x86/dsputil_yasm.asm:1130: error: instruction expected after label
libavcodec/x86/dsputil_yasm.asm:1130: error: instruction expected after label
libavcodec/x86/dsputil_yasm.asm:1130: error: instruction expected after label
libavcodec/x86/dsputil_yasm.asm:1130: error: instruction expected after label
libavcodec/x86/dsputil_yasm.asm:1130: error: instruction expected after label
libavcodec/x86/dsputil_yasm.asm:1130: error: undefined symbol `CLIPD_SSE41.loop' (first use)
make: * [libavcodec/x86/dsputil_yasm.o] Error 1

I am currently using openSUSE, and YASM version is :

# yasm --version
yasm 1.1.0.2352

Finally I found that the problem was commit 38e06c2, where clipd macros have moved from libavcodec/x86/dsputil_yasm.asm to libavutil/x86/x86util.asm :

commit 38e06c2969184b5b55ec41d0c053b2480ab52846
Author: Ronald S. Bultje <​rsbultje@gmail.com>
Date: Tue Aug 16 21:23:53 2011 -0700

Move clipd macros to x86util.asm.

This allows sharing them between multiple .asm files.

libavcodec/x86/dsputil_yasm.asm | 34 +---------------------------------
libavutil/x86/x86util.asm | 33 +++++++++++++++++++++++++++++++++
2 files changed, 34 insertions(+), 33 deletions(-)

Having a look at the code, i can't see anything wrong; macro have moved, but an include has been properly added in dsputil_yasm.asm.
However, i am not a YASM expert but i guess this is a YASM bug.

Up to now i have simply reverted 38e06c2, since noone seemed to use this common part elsewhere in the code.

[Carl Eugen Hoyos]

Please confirm that you tried the following:
$ make distclean
$ git checkout master
$ git pull
$ git diff (shows no differences)
$ ./configure

Then run
$ make V=1 libavcodec/x86/dsputil_yasm.o
if it does not succeed, please post the complete output, the first line of git log and attach config.asm

[Gilles Sabourin]

The generation succeded!

$ make V=1 libavcodec/x86/dsputil_yasm.o
touch .version
yasm -f elf -m amd64 -g dwarf2 -I. -I./ -I./libavutil/x86/ -Pconfig.asm -I libavcodec/x86/ -M -o libavcodec/x86/dsputil_yasm.o libavcodec/x86/dsputil_yasm.asm > libavcodec/x86/dsputil_yasm.d
yasm -f elf -m amd64 -g dwarf2 -I. -I./ -I./libavutil/x86/ -Pconfig.asm -I libavcodec/x86/ -o libavcodec/x86/dsputil_yasm.o libavcodec/x86/dsputil_yasm.asm
$ ll libavcodec/x86/dsputil_yasm.o
-rw-r--r-- 1 gilles users 28016 11 sept. 14:31 libavcodec/x86/dsputil_yasm.o

[Gilles Sabourin]

Replying to cehoyos:
Can you explain a bit more ? This problem is totally reproductible on my 64 bits PCs and also in a virtual box 32 bits virtual machine, both using openSUSE 11.4

Moreover, i generate packages automatically using spec file. Hence I always start with a clean source folder.

[Carl Eugen Hoyos]

You wrote "The generation succeded!"

You will have to find out what makes it fail for your other configuration (I don't mean the version introducing it, but what in your build chain makes the difference, I also tried with your configure line as far as I could use it).

[Gilles Sabourin]

Finally! I found that a part the problem is a difference that lies in libavcodec/x86/dsputil_yasm.d content.

The test below works, and the content of dsputil_yasm.d is :

libavcodec/x86/dsputil_yasm.o: libavcodec/x86/dsputil_yasm.asm config.asm \

./libavutil/x86/x86inc.asm ./libavutil/x86/x86util.asm

As you can see, asm files are prefixed by "./"

The generation fails in my case, and i can see that the content of dsputil_yasm.d is :
libavcodec/x86/dsputil_yasm.o: libavcodec/x86/dsputil_yasm.asm config.asm \

libavcodec/x86/x86inc.asm libavcodec/x86/x86util.asm

and asm files are NOT prefixed by "./"

The second case appends when you use a build system like rpm. Also I am doing generation under root account. Normally you begin by something like : cd /usr/src/package/BUILD ; cd ffmpeg
and that should be enough to have relative paths from top source directory.

Do you want to reproduce the problem ? Simply put a comment at the include in libavcodec/x86/dsputil_yasm.asm (line number 23) :

;* %include "x86util.asm"

gilles@gilles-portable:~/ffmpeg-src> make V=1 libavcodec/x86/dsputil_yasm.o
touch .version
yasm -f elf -m amd64 -g dwarf2 -I. -I./ -I./libavutil/x86/ -Pconfig.asm -I libavcodec/x86/ -M -o libavcodec/x86/dsputil_yasm.o libavcodec/x86/dsputil_yasm.asm > libavcodec/x86/dsputil_yasm.d
yasm -f elf -m amd64 -g dwarf2 -I. -I./ -I./libavutil/x86/ -Pconfig.asm -I libavcodec/x86/ -o libavcodec/x86/dsputil_yasm.o libavcodec/x86/dsputil_yasm.asm
libavcodec/x86/dsputil_yasm.asm:1122: error: instruction expected after label
libavcodec/x86/dsputil_yasm.asm:1122: error: instruction expected after label
libavcodec/x86/dsputil_yasm.asm:1122: error: instruction expected after label
libavcodec/x86/dsputil_yasm.asm:1122: error: instruction expected after label
libavcodec/x86/dsputil_yasm.asm:1122: error: undefined symbol `CLIPD_MMX.loop' (first use)
libavcodec/x86/dsputil_yasm.asm:1122: error: (Each undefined symbol is reported only once.)
libavcodec/x86/dsputil_yasm.asm:1125: error: instruction expected after label
libavcodec/x86/dsputil_yasm.asm:1125: error: instruction expected after label
libavcodec/x86/dsputil_yasm.asm:1125: error: instruction expected after label
libavcodec/x86/dsputil_yasm.asm:1125: error: instruction expected after label
libavcodec/x86/dsputil_yasm.asm:1127: error: instruction expected after label
libavcodec/x86/dsputil_yasm.asm:1127: error: instruction expected after label
libavcodec/x86/dsputil_yasm.asm:1127: error: instruction expected after label
libavcodec/x86/dsputil_yasm.asm:1127: error: instruction expected after label
libavcodec/x86/dsputil_yasm.asm:1127: error: instruction expected after label
libavcodec/x86/dsputil_yasm.asm:1127: error: instruction expected after label
libavcodec/x86/dsputil_yasm.asm:1127: error: instruction expected after label
libavcodec/x86/dsputil_yasm.asm:1127: error: instruction expected after label
libavcodec/x86/dsputil_yasm.asm:1127: error: undefined symbol `CLIPD_SSE2.loop' (first use)
libavcodec/x86/dsputil_yasm.asm:1130: error: instruction expected after label
libavcodec/x86/dsputil_yasm.asm:1130: error: instruction expected after label
libavcodec/x86/dsputil_yasm.asm:1130: error: instruction expected after label
libavcodec/x86/dsputil_yasm.asm:1130: error: instruction expected after label
libavcodec/x86/dsputil_yasm.asm:1130: error: instruction expected after label
libavcodec/x86/dsputil_yasm.asm:1130: error: instruction expected after label
libavcodec/x86/dsputil_yasm.asm:1130: error: instruction expected after label
libavcodec/x86/dsputil_yasm.asm:1130: error: instruction expected after label
libavcodec/x86/dsputil_yasm.asm:1130: error: undefined symbol `CLIPD_SSE41.loop' (first use)
make: * [libavcodec/x86/dsputil_yasm.o] Erreur 1

This looks like YASM is not able to find libavcodec/x86/dsputil_yasm.asm to include an fails. Maybe an explicit path is missing in makefile ?

[Carl Eugen Hoyos]

Please remove the old x86util.asm and x86inc.asm files from libavcodec/x86

[Gilles Sabourin]

Replying to cehoyos:

Please remove the old x86util.asm and x86inc.asm files from libavcodec/x86

Please stop closing the bug report every time. Your answer told me you haven't read well comment #6.
I have NO old .asm stuff since I rebuild EVERY time from SCRATCH with an rpm build system.
I am trying to explain to you :
1 - why i get some differences between 2 generated files
2 - that the errors are SIMILAR when you comment an include in libavcodec/x86/dsputil_yasm.asm

Conclusion : There is a corner case in makefile where YASM can't find the include .asm file

[Carl Eugen Hoyos]

Replying to sabouring:

The generation fails in my case, and i can see that the content of dsputil_yasm.d is :
libavcodec/x86/dsputil_yasm.o: libavcodec/x86/dsputil_yasm.asm config.asm \

libavcodec/x86/x86inc.asm libavcodec/x86/x86util.asm

which shows you have x86inc.asm and x86util.asm in libavcodec/x86.
Both files do not belong there.
(I agree that there might be a bug in dsputil_yasm.asm since it does not use a complete path for including x86*.asm, but I suspect changing this might break things because of how yasm wants its include paths.)

Please reopen this ticket if you can reproduce it without using rpm and without changing sources.

[Gilles Sabourin]

without using rpm ? Well, you'll have to explain that to packman packagers. Now, they'll have to patch only 2 files from git-master, but this may end up with more complicated patchs if the common part is reused elsewhere in ffmpeg code.