Written mp4 with h264/AAC doesnt use fps as video timebase (discrepancy with libav)

[Lastique]

In our application, we write mp4 files through libav 9.6 using its public API. In one particular case we mux h264 video and AAC audio into the file, so the code is roughly like this:

AVFormatContext* m_av_format_ctx = ...;

AVCodec* video_codec = NULL;
AVCodec* audio_codec = NULL;

CodecID codec_id = AV_CODEC_ID_H264;
video_codec = avcodec_find_encoder(codec_id);

m_av_format_ctx->oformat->video_codec = codec_id;
m_av_format_ctx->video_codec_id = codec_id;
AVStream* av_stream = avformat_new_stream(m_av_format_ctx, NULL);
av_stream->id = 0;

av_stream->r_frame_rate = av_d2q(m_video_info->fps, std::numeric_limits< unsigned short >::max());
av_stream->time_base.num = av_stream->r_frame_rate.den;
av_stream->time_base.den = av_stream->r_frame_rate.num;

av_video_codec_ctx = av_stream->codec;
// av_stream->codec initialization skipped...
av_video_codec_ctx->codec_type = AVMEDIA_TYPE_VIDEO;
av_video_codec_ctx->flags |= CODEC_FLAG_GLOBAL_HEADER;
av_video_codec_ctx->time_base = { 1, 1000 };


codec_id = AV_CODEC_ID_AAC;
audio_codec = avcodec_find_encoder(codec_id);

m_av_format_ctx->oformat->audio_codec = codec_id;
m_av_format_ctx->audio_codec_id = codec_id;

AVStream* av_stream = avformat_new_stream(m_av_format_ctx, NULL);
av_stream->id = 1;

av_audio_codec_ctx = av_stream->codec;
av_stream->time_base.num = 1;
av_stream->time_base.den = m_audio_info->sample_rate;
av_audio_codec_ctx->codec_type = AVMEDIA_TYPE_AUDIO;
av_audio_codec_ctx->flags |= CODEC_FLAG_GLOBAL_HEADER;
av_audio_codec_ctx->time_base = { 1, 1000 };

(Sorry, I cannot post the complete code)

Our internal video and audio timestamps are in milliseconds, but for audio we convert them to the sample rate-based values by calling av_rescale_rnd(). Video timestamps are converted to frame number (since for fixed-fps media we must specify time_base as 1/fps).

The files are written ok by libav 9.6 this way, but not by ffmpeg 1.2.1 (our code is unchanged). Files written by ffmpeg are not playable, video frame rate is extremely low (about 1 frame per several seconds). I have attached files written by libav and ffmpeg.

[Lastique]

File created by libav 9.6. Plays ok in mplayer, ffplay or vlc.

[Lastique]

File created by ffmpeg 1.2.1. Doesn't play in mplayer, ffplay or vlc.

[Carl Eugen Hoyos]

Please also test current git head.

Is the problem also reproducible with ffmpeg (the application) or only if you use library calls from your application?

[Hendrik]

Note that it is possible for avformat to change the stream timebase from the one you specified when it thinks that another one is more appropriate for the format, so you should always rescale your timestamps to the stream timebase, or your assumption may go wrong.

[Lastique]

Replying to cehoyos:

Please also test current git head.

The latest snapshot behavior is the same.

Is the problem also reproducible with ffmpeg (the application) or only if you use library calls from your application?

We only use the library API in our application, and I'm not sure how I can reproduce the same library setup from ffmpeg command line.

Replying to heleppkes:

Note that it is possible for avformat to change the stream timebase from the one you specified when it thinks that another one is more appropriate for the format, so you should always rescale your timestamps to the stream timebase, or your assumption may go wrong.

Yes, this seems to be the problem. libav does not modify time_base we set in the context initialization, and ffmpeg sets video time_base to be equal to the audio time_base. If we use the time_base set by ffmpeg for rescaling then the written file is playable. Thanks for the suggestion.

Frankly, this is an odd feature. Why would the library change settings requested by the caller?

[Michael Niedermayer]

Replying to Lastique:

Replying to cehoyos:

Please also test current git head.

The latest snapshot behavior is the same.

Is the problem also reproducible with ffmpeg (the application) or only if you use library calls from your application?

We only use the library API in our application, and I'm not sure how I can reproduce the same library setup from ffmpeg command line.

Replying to heleppkes:

Note that it is possible for avformat to change the stream timebase from the one you specified when it thinks that another one is more appropriate for the format, so you should always rescale your timestamps to the stream timebase, or your assumption may go wrong.

Yes, this seems to be the problem. libav does not modify time_base we set in the context initialization, and ffmpeg sets video time_base to be equal to the audio time_base. If we use the time_base set by ffmpeg for rescaling then the written file is playable. Thanks for the suggestion.

Frankly, this is an odd feature. Why would the library change settings requested by the caller?

To ensure the timebase is precisse enough to allow precissely specifying timestamps of frames. Also some containers like mpeg-ps/ts use a fixed 90khz timebase that cannot be changed.
Also note, the mov/mp4 muxers have an option to force a specific video timebase (video_track_timescale) if you want a specific course grained one.

[Cigaes]

See the following commit, fixing ticket #2892:
​http://git.videolan.org/?p=ffmpeg.git;a=commitdiff;h=2501f6d3d6b6e5db58ff4756baf52c69c92f096c
Apparently, the time base is forced by the muxer specifically to support the "time code track" feature. I have no idea what actual use it is.

[Lastique]

To ensure the timebase is precisse enough to allow precissely specifying timestamps of frames. Also some containers like mpeg-ps/ts use a fixed 90khz timebase that cannot be changed.

I think, that's the container details libavformat is supposed to abstract away from the user. Timestamp conversion is performed in any case - either inside the muxer or in the user's code, when it discovers what time base the muxer wants. My point is that the muxer is a better place for the conversion since it eases life for the user and makes libavformat interface more uniform across different containers.

You could report the "desired" time base through some new interface before the user opens the context so that when the user has the liberty of choosing his internal timestamp granularity he can use the time base that is best for the container. In this case the muxer wouldn't have to perform the conversion.