sbgdec: replace EOVERFLOW with ENOMEM.
This should fix track issue #781, regarding compilation with i686-w64-mingw32-gcc.