ffmpeg does not mark xsub language properly

[someuser]

Summary of the bug:
When creating AVI/DivX with XSub subtitles using FFmpeg the language for subtitles is not marked properly unless the .idx file happens to contain the needed "alt" tag which might be very hard to discover for casual users.

When using AVIAddXSubs or an .idx file with an alt tag with FFmpeg the language of subtitles is recognized properly by player like PS3 with format (below is examples for English and French):

alt: Subtitle - en-xx;02
alt: Subtitle - fr-xx;02

I've verified with PS3 that this format works with more than one subtitle streams.

It would seem that the optimal solution would be to

1) Use the alt tag from the .idx file if present (=this is what happens today)

2) If the user supplies something like -metadata:s:s:0 language=eng then it should be used (note that the lang codes are 2 chars with XSubs)

3) If there is no alt tag in the .idx file and no metadata options provided in the command line then then add one based on the above format and the language code specified by the id tag in the .idx

How to reproduce:
If one creates AVI/DivX with XSubs using FFmpeg without the alt tag in the .idx file some player like PS3 present the language of the subtitles as "Other". When using the syntax about PS3 detects the language correctly AND automatically loads the subtitles if a subtitle for language set as default in the player settings is found. Currently, using any metadata command line parameters does not provide the necessary language information being added.

[Carl Eugen Hoyos]

Please provide a sample (one with correct language code) and a command line that incorrectly removes the language information together with complete, uncut console output.

[someuser]

"a command line that incorrectly removes the language information together with complete, uncut console output"

Hmm, what do you mean with this? The language information is not *added* (at least so that PS3 would recognize it) at all unless the "alt" tag in the proper format is not in the .idx file.

I'll attach a 10 sec sample AVI with subtitles recognized by PS3 shortly.

Thanks.

[Hendrik]

He needs the commandline you used to transcode with ffmpeg, the one that creates the improper files. And in addition, the output of ffmpeg when doing so.

[someuser]

Sorry, forgot the command line + output, here it is:

> ./ffmpeg-git-17c1881-win32-static.exe -i test.avi -i test-en.idx -acodec copy -vcodec copy -scodec xsub -map 0:0 -map 0:1 -map 1:0 testxsub.avi
ffmpeg version N-51106-g17c1881 Copyright (c) 2000-2013 the FFmpeg developers
  built on Mar 19 2013 13:49:37 with gcc 4.7.2 (GCC)
  configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-libass --enable-libbluray --enable-libcaca --enable-libfreetype --enable-libgsm --enable-libilbc --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libschroedinger --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvo-aacenc --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libxavs --enable-libxvid --enable-zlib
  libavutil      52. 19.101 / 52. 19.101
  libavcodec     55.  1.100 / 55.  1.100
  libavformat    55.  0.100 / 55.  0.100
  libavdevice    55.  0.100 / 55.  0.100
  libavfilter     3. 47.104 /  3. 47.104
  libswscale      2.  2.100 /  2.  2.100
  libswresample   0. 17.102 /  0. 17.102
  libpostproc    52.  2.100 / 52.  2.100
Input #0, avi, from 'test.avi':
  Metadata:
    encoder         : Lavf54.59.106
  Duration: 00:00:10.03, start: 0.000000, bitrate: 1227 kb/s
    Stream #0:0: Video: mpeg4 (Advanced Simple Profile) (DIVX / 0x58564944), yuv420p, 576x324 [SAR 1:1 DAR 16:9], 23.98 fps, 23.98 tbr, 23.98 tbn, 2997 tbc
    Stream #0:1: Audio: mp3 (U[0][0][0] / 0x0055), 48000 Hz, stereo, s16p, 128 kb/s
[vobsub @ 0259f3e0] Estimating duration from bitrate, this may be inaccurate
Input #1, vobsub, from 'test-en.idx':
  Duration: N/A, bitrate: N/A
    Stream #1:0[0x0](en): Subtitle: dvd_subtitle (default)
Output #0, avi, to 'testxsub.avi':
  Metadata:
    ISFT            : Lavf55.0.100
    Stream #0:0: Video: mpeg4 (DIVX / 0x58564944), yuv420p, 576x324 [SAR 1:1 DAR 16:9], q=2-31, 23.98 fps, 23.98 tbn, 23.98 tbc
    Stream #0:1: Audio: mp3 (U[0][0][0] / 0x0055), 48000 Hz, stereo, 128 kb/s
    Stream #0:2(en): Subtitle: xsub (DXSB / 0x42535844) (default)
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #0:1 -> #0:1 (copy)
  Stream #1:0 -> #0:2 (dvdsub -> xsub)
Press [q] to stop, [?] for help
frame=  240 fps=0.0 q=-1.0 Lsize=    1511kB time=00:00:10.03 bitrate=1233.7kbits/s
video:1322kB audio:157kB subtitle:3 global headers:0kB muxing overhead 2.000717%
> 

As said, this plays ok on PS3 and the subtitles can be manually selected but the language shown is "Other" and PS3 won't pick them up automatically (as it would happen if the language would have been detected and it would match PS3 settings).

[someuser]

This is what mediainfo 0.7.61 prints about the subtitle stream created without the alt tag:

Text
ID                                       : 2
Format                                   : DivX Subtitle
Codec ID                                 : DXSB
Codec ID/Info                            : Subtitle in AVI from DivX networks
Width                                    : 576 pixels
Height                                   : 324 pixels
Bit depth                                : 24 bits
Stream size                              : 2.82 KiB (0%)

And here is mediainfo output when "alt: Subtitle - en-xx;02" was present:

Text
ID                                       : 2
Format                                   : DivX Subtitle
Codec ID                                 : DXSB
Codec ID/Info                            : Subtitle in AVI from DivX networks
Width                                    : 576 pixels
Height                                   : 324 pixels
Bit depth                                : 24 bits
Stream size                              : 2.82 KiB (0%)
Title                                    : Subtitle - en-xx;02

[Carl Eugen Hoyos]

Is the problem also reproducible if you stream-copy or reencode an avi file that contains subtitles with correct language codes?

[Carl Eugen Hoyos]

Does textxsub.avi - the file you attached - contain working language codes?

[someuser]

Replying to cehoyos:

Does textxsub.avi - the file you attached - contain working language codes?

No, it is what is produced without the alt tag. I'll attach now testxsub-ok.avi which contains the working language code.

Thanks.

[Cigaes]

Attaching your vobsub files, i.e. the input of the failing command lines, would be more helpful than attaching their output.

[someuser]

Ok, I'll also attach the .idx/.sub files, the .idx file will contain the alt tag commented out.

[Cigaes]

The difference of treatment between the two cases seems to boil down to this:

$ diff -u <(ffprobe -loglevel 0 -show_streams test-en.idx) <(ffprobe -loglevel 0 -show_streams test-enxx.idx)
--- /proc/self/fd/11    2013-03-20 13:45:42.411422486 +0100
+++ /proc/self/fd/13    2013-03-20 13:45:42.411422486 +0100
@@ -31,4 +31,5 @@
 DISPOSITION:clean_effects=0
 DISPOSITION:attached_pic=0
 TAG:language=en
+TAG:title=Subtitle - en-xx;02
 [/STREAM]

If I understand correctly the issue, it seems that the language is only recognized when the "title" metadata tag is present.

And indeed, the "title" metadata tag arrives to the AVI file, while the "language" metadata tag is completely ignored.

The core problem looks like it is: the AVI muxer is ignoring the "language" metadata tag.

[Carl Eugen Hoyos]

Fixed by Erik Olofsson in f5bffd3a

[someuser]

Thanks, I'm happy to confirm the patch fixed the issue!