Encoded movies with mov_text subtitles do not play with QT Player

[Atarikid]

When a .mkv contains subrip subtitle(s) and you encode to mp4 with -c:s mov_text the subtitles do not work when using the QuickTime Player (OSX). It plays fine with VLC though.

When encoding with Handbrake (which uses mov_text for mp4) it works fine with QuickTime Player.

FFmpeg commandline:
/Users/atarikid/Desktop/ffmpeg -i /Volumes/Data/Movies/Homeland?.mkv -c:a aac -c:v libx264 -strict -2 -c:s mov_text /Users/atarikid/Desktop/test.mp4

[Carl Eugen Hoyos]

Could you point us to a file with mov_text that is played correctly with QuickTime?

[Atarikid]

I have uploaded two encoded movies.
One with FFmpeg and commandline:
/Users/atarikid/Desktop/ffmpeg -i /Volumes/Data/Movies/Homeland??.mkv -c:a aac -c:v libx264 -strict -2 -c:s mov_text /Users/atarikid/Desktop/test.mp4

The subtitles do not work with QuickTime Player (main OSX player). Works with VLC

The second is encoded with Handbrake (with mov_text):
Subtitles work fine with QuickTime Player and VLC

[Atarikid]

It is note worthy that with QuickTime Player you cannot even select the subtitles stream. So maybe the header created by FFmpeg is wrong?

[Carl Eugen Hoyos]

I tested "Handbrake-subtitles work with QTPlayer.mp4" with QuickTime 7.7.2 and it shows no subtitles here. Which version did you test?

[Atarikid]

Tested with QuickTime Player 10.2 (OS X 10.8 and OS X 10.7).
You first have to choose the subtitles in the menu. By default the subtitles are disabled in QT Players.

BTW QT 7.7.2 is a tremendous old version no-one uses with OSX. :)

[Carl Eugen Hoyos]

Replying to Atarikid:

BTW QT 7.7.2 is a tremendous old version no-one uses with OSX. :)

Not everybody owns a Mac...

[Atarikid]

Besides being the most irritating answer ever, what does this answer contributes to the query?

[Carl Eugen Hoyos]

I told you I tested QuickTime 7.7.2 (this was current at the time of my writing on Windows, it's now 7.7.3 iirc) and was unable to reproduce the issue, you answered that my version of QuickTime is outdated (which is true, I wasn't aware of it though). It took me some time to find a Mac that allows to reproduce the problem (on my Mac, 7.7.2 is also the latest version) and I wanted to explain why it took me so long to reproduce the issue.

Long-time experience tells me that tickets that cannot be easily reproduced are not easily fixed (in this case it would have been possible imo that somebody who does want to work on it would have been unable to reproduce the problem exactly as I was unable to).

[Atarikid]

OK, sorry for my reply. But only 'Not everybody owns a Mac' came out a bit rude imo :-)

Anyhow, glad you could replicate this issue. FFmpeg is quit common on Mac OS X (many converters use it). So making it work for QT Player is imo important.

[Carl Eugen Hoyos]

Replying to Atarikid:

OK, sorry for my reply. But only 'Not everybody owns a Mac' came out a bit rude imo :-)

That is exactly the feeling I had reading "QT 7.7.2 is a tremendous old version no-one uses"

[Carl Eugen Hoyos]

There is another (older) sample - ​http://samples.ffmpeg.org/mov/subtitles-embedded/BCDisassembly1RightSideTxt2.mov - that plays fine with QT 7.7.3
It works fine with QT if the subtitle stream is copied (-scodec copy), shows weird (enlarged, red, misplaced) subtitles on QT with -scodec mov_text

[Philip Langdale]

It sounds like the player is, at least to some extent, respecting the text formatting and positioning flags in the header - which no other player I'm aware of does. That's why I was never able to test this.

We used a fixed style which should yield "12 point Serif white-on-transparent" with undefined positioning. This could easily lead to invisible subtitles due to the lack of positioning information, or misplaced subtitles (top left corner?).

The only thing that really surprises me is the red colour. I know I didn't mess that up.

[Carl Eugen Hoyos]

Replying to philipl:

It sounds like the player is, at least to some extent, respecting the text formatting and positioning flags in the header - which no other player I'm aware of does.

Excuse my ignorance: What other player than QuickTime supports mov_text in mov?

[Philip Langdale]

mplayer, vlc - all the usual suspects. Am I missing something?

And iOS devices, although also made by Apple use very different playback software than Quicktime on the desktop.

[Carl Eugen Hoyos]

I tried the following with the sample attached to this ticket (that plays subtitles on my iPhone):
$ ffmpeg -i Handbrake-subtitles\ work\ with\ QTPlayer.mp4 -strict -2 -qscale 5 -scodec mov_text outmovtext.mov
The resulting file shows no subtitles on my iPhone.

[Philip Langdale]

The situation is confusing, to say the least.

On my ipad: both the attached samples work correctly.

When I remux the Handbrake sample, I get a file that's identified as having subtitles, but the subtitle duration is way off, and the subtitles 'stick' on screen for too long - I think there's some bug in how the time base is being translated. But the subtitles, are nevertheless, being transcoded and shown in the ipad.

I'm surprised to see a significant difference between ipad and iphone.

[Philip Langdale]

Also note that my windows QT 7.7.3 reports the attached Handbrake sample as having no subtitles. So I really don't know what's going on right now.

[Carl Eugen Hoyos]

Replying to philipl:

When I remux the Handbrake sample

I did not test remuxing but re-encoding the subtitles with -scodec mov_text

[Carl Eugen Hoyos]

Replying to philipl:

Also note that my windows QT 7.7.3 reports the attached Handbrake sample as having no subtitles.

It works fine with newer QT, see above.

[Philip Langdale]

So, there are multiple things going on here that affect the success, or otherwise, of this process.

1) File type subtleties

There are meaningful differences between mov/mp4/ipod output types as generated by the mov encoder. I always use 'ipod' and I've succeeded in producing files with subtitle tracks that are recognised by all tested players (QT, iPad, mplayer) this way. But not always. I've got input samples that don't seem to yield subtitles recognised by QT even with the same command line. Don't get it. As mentioned above, there may be some subtitle timebase translation failure going on when remuxing existing mp4 files.

2) The definite feature gaps

a) The width/height and positional offset of the text track need to be set to meaningful values by the mov encoder (this header is not handled by the mov_text encoder).

b) The text box (which exists within the text track) needs to be sized to meaningful values by the mov_text encoder.

In practice, these values are closely related and in turn closely related to the width/height of the video track. I have no idea how one would sanely set these values in a file with multiple video tracks and/or ones with varying sizes.

A rough heuristic that works for a single constant size video is that the text track has the same width as the video, and some 'reasonable' height in pixels - which we might be able to derive as a percentage of the video height. The vertical offset of the text track is then video height - text height - assuming we want the subtitle area to be aligned exactly with the bottom of the video area. (It is legal to offset outside the video area and the quicktime player will make this work)

The text box is then 'normally' sized to the dimensions of the text track (ie: fills the whole track). This means that the mov_text encoder cannot know ahead of time what a reasonable size for the text box is. We'd really have to make the mov encoder re-write these values when writing the track header.

This is all massively overengineered and very hard to get right, because - of course, subtitle size and display area really should be controlled by the playback device and not by the file encoder!

[Philip Langdale]

Oh, and to state it explicitly: by hard-coding various sizes and offsets in the encoders, I was able to produce a file with subtitles that rendered correctly in QT 7.3.3 on windows and OSX Player 10.1 (what I was able to test with on a Mac I could fine).

So, the fundamental encoding, with respect to atom types and formatting is correct. But in the absence of accurate sizing information, it's not going to display correctly.

[Carl Eugen Hoyos]

Is the sizing information something the encoder or the muxer has to write? Does the muxer have enough information?

[Philip Langdale]

When I say "mov encoder", I mean the muxer.

The muxer is the *only* thing that has enough information, because it's the only thing that knows about the video tracks that the subtitles will be used with.

[Nick]

for additional comments and example files see also:
https://ffmpeg.org/trac/ffmpeg/ticket/2488

[Clément Bœsch]

Is this still true? It's potentially fixed by 30ce2890.

[eisa01]

The bug is still there for me, assuming I compiled it correctly

(cloned the git repository and ran configure, make, make install and verified the new compilation date of ffmpeg)
ffmpeg version N-55736-gb329ff3 Copyright (c) 2000-2013 the FFmpeg developers

built on Aug 23 2013 23:40:45 with Apple LLVM version 4.2 (clang-425.0.28) (based on LLVM 3.2svn)

[Philip Langdale]

This bug's not going to go away without either a bunch of work in the muxer to translate video track sizing information into subtitle track sizing, or adding a bunch of encode/mux options that expose all the silly box dimension settings. Based on feedback when I started discussing this many months ago, the second option is preferred but is trickier to execute than it looks.

[Philip Langdale]

During GSoC this year, my intern implemented support for most of the text formatting capabilities in timed text, including applying a non-trivial default style when encoding. He didn't do anything with display area positioning and sizing, but nevertheless, we now observe that, at least on OSX, the QT Player is happily showing subtitles at a reasonable size in a reasonable position. I suspect this is due to the default style being more meaningful, and triggering some default positioning logic in the QT Player.

So, certainly this is fixed on OSX. My intern said he was not able to see any subtitles on windows with or without his changes. That still needs investigating, but it's a separate issue, so I'm going to mark this as closed.

[Avi Alkalay]

This is still not working as of ffmpeg 3.3.6

Files with subtitles that work on iOS and macOS (created by gpac MP4Box) show the following on ffprobe:

Stream #0:2(eng): Subtitle: mov_text (tx3g / 0x67337874), 720x1280, 0 kb/s (default)

While files created by ffmpeg show the following:

Stream #0:2(eng): Subtitle: mov_text (text / 0x74786574), 0 kb/s (default)

And I can't find a way to tweak ffmpeg command line to force txg3.

[Avi Alkalay]

This is the solution and it works on my macOS and iOS devices, but this is the first place where is documented:

ffmpeg -i video.mov -canvas_size 1920x1080 -i sub.srt -c copy -c:s mov_text -tag:s:s:0 tx3g new.mov

The -tag:s:s:0 tx3g does the trick. The -canvas_size WxH doesn't seem necessary but that is what make files look similar, in the metadata level, to what gpac MP4Box generates.

Also, resulting file must be named .mov, not .mp4 and not .m4v, otherwise a different file brand will be set which makes ffmpeg complain.