H264/AAC MP4 video first frame doesn't report a zero mediaTime using HTML requestVideoFrameCallback API

[Daniel]

Version: 6.0-essentials_build-www.gyan.dev or N-109421-g9adf02247c-20221216

Platform: Windows 10 (Browser: Chrome 113.0.5672.92)

Summary: Mp4 files generated using ffmpeg aren’t reporting a mediaTime of 0 via requestVideoFrameCallback for the first frame. This behavior is not observed with files encoded as webm or via another encoder such as MainConcept.

Description:
When incrementing frames with video 1, progress is not visible until the third button press in the demo provided here: ​https://randomstuffdr.s3.eu-west-1.amazonaws.com/TEST-v3.HTML

The mediaTime issue may be due to the aac encoder in ffmpeg, as the problem does not occur when using aac_mf.

This might be related to how Chromium implements mediaTime, using the audio clock as the time source that backs video.currentTime, while the mediaTime is directly populated by the presentationTimestamp of the frame, as noted here: ​https://web.dev/requestvideoframecallback-rvfc/

Actual results:

No video progress is observed until the third button press.

Expected results:

Video progress should be visible from the first frame.

Additional information:

Attempts to resolve the issue with ffmpeg (disabling b-frames, use baseline profile and avoid_negative_ts) have not been successful.

Question:

Is there a way to generate a file in a similar manner to MainConcept or ffmpeg when using acc_mf?

Commands used to generate files
Video 1 – (Video: H264 /Audio: AAC / Wrapper: MP4 / Transcoder: FFMPEG) .\ffmpeg.exe -i '720p50 Flash and Beep.mxf' -map 0:0 -map 0:1 -map_metadata -1 -c:v
libx264 -profile:v baseline -level:v 2.0 -pix_fmt yuv420p -crf 28 -b:v 118k -maxrate 118k -bufsize 236k -vf scale="480:-1" -preset veryfast -c:a aac "..\low-res\720p50FlashandBeep_V_A.mp4"

Video 2 – (Video: H264 /Audio: AAC / Wrapper: MP4 / Transcoder: FFMPEG) .\ffmpeg.exe -i '720p50 Flash and Beep.mxf' -map 0:0 -map 0:1 -map_metadata -1 -c:v
libx264 -profile:v baseline -level:v 2.0 -pix_fmt yuv420p -crf 28 -b:v 118k -maxrate 118k -bufsize 236k -vf scale="480:-1" -preset veryfast -c:a aac_mf "..\low-res\720p50FlashandBeep_V_A_mf.mp4"

Video 3 – (Video: H264 /Audio: No Audio / Wrapper: MP4 / Transcoder: FFMPEG) .\ffmpeg.exe -i '720p50 Flash and Beep.mxf' -map 0:0 -map 0:1 -map_metadata -1 -c:v
libx264 -profile:v baseline -level:v 2.0 -pix_fmt yuv420p -crf 28 -b:v 118k -maxrate 118k -bufsize 236k -vf scale="480:-1" -preset veryfast -an "..\low-res\720p50FlashandBeep_V.mp4"

Video 4 – (Video: vp8 /Audio: ogg / Wrapper: webm / Transcoder: FFMPEG) .\ffmpeg.exe -i '720p50 Flash and Beep.mxf' -map 0:0 -map 0:1 -map_metadata -1 -c:v vp8 -crf 28 -b:v 118k -maxrate 118k -bufsize 236k -vf scale="480:-1" -preset veryfast "..\low-res\720p50FlashandBeep_V_A.webm"

Video 5 – (Video: H264 /Audio: AAC / Wrapper: MP4 / Transcoder: MainConcept)