-y option affects audio/video skew during screen capture (alsa+x11grab)

[grepfor]

Example: On my system, the following script works fine and consistently renders normally synchronized audio and video:

ffmpeg  -f alsa -ac 2 -i hw:0 \
        -f x11grab -r 30 -s 1600x900 -i :0.0 \
        -acodec mp3 \
        -vcodec libx264 -preset ultrafast -threads 0 \
        -f matroska -y ofile.mkv

However, removing the -y option -- and assuming that a file named ofile.mkv already exists, so that ffmpeg asks whether to overwrite it -- results in skew between audio and video.

The observed skew seems to be roughly equal to the time it takes for the user to respond to the File exists. Overwrite? question. This suggests that the source of the problem may be that the audio timestamp is being initialized prior to the Overwrite? question and the video timestamp initialized after the user response. Just guessing. In any case, omitting -y produces gross skew effects, and is 100% reproducible on my system.

Logfiles obtained using the -report option are attached, one for each case (with and without -y option).

NOTE: This ticket was filed with priority "important" because during my own investigation of the issue, I came across a large number of reports by others also stumped by A/V skew peculiarities when attempting aud+vid screencaps using ffmpeg. Many were frustrated at being unable to get any of the (numerous) cargo-cult 'recipe' commandline approaches to work. A few of the recipies do explicitly specify using -y, but I didn't see any explanations as to why it was necessary, and it seems (at least to me) extremely counterintuitive that that option should have any such effect. In short, aud+vid screencap seems like a not-uncommon application of ffmpeg and fixing this sooner rather than later may help quite a few people avoid tripping over the same issue.

Just my 2c, and perhaps exaggerating the need for priority. Obviously re-prioritize as you see fit.

NOTES:

0. Demo system was Arch linux, synched to repos about 2 days ago.
1. Effect has been observed using mp3 and pcm_s16le audio codecs.
2. Have not tried it with video codecs other than libx264.

[grepfor]

Log without -y option

[grepfor]

Log with -y option

[Carl Eugen Hoyos]

Please mention if this is a regression and please test current FFmpeg git head.

[grepfor]

Not sure if it's a regression or not. I've not seen it previously, but also have never attempted to use ffmpeg in this way previously either. However, based on cargo cult postings from several years ago, a few of which seem to imply that -y is somehow important when doing screen capture, I would guess -- and it's only a guess -- that it is a long-standing issue.

Unfortunately, I do not have time to build it from git sources. However, ffmpeg-git r60910.1f2bacc-1 is available from the Arch repos, which I can obtain and run easily as a package install. Will that be of any use to you to know the results from that version?

[grepfor]

Please update 'Version' field in ticket header to 2.2.2. I just tried it with that version from the Arch repos, and the problem still occurs.

Reproducibility note: On my system, this issue is 100% reproducible, and the resulting a/v skew is extremely obvious, especially if you wait several seconds before responding to the Overwrite? query. So the effort required for a developer to attempt reproducing it on up-to-date git code should be on the order of 1-2 minutes, literally. Just cut-n-paste the script, remove -y, and pre-create a file called ofile.mkv so that ffmpeg asks about overwriting it. I suspect you will see the problem immediately. If not, let me know.

[Timothy Pearson]

I am able to confirm this bug exists in the latest FFmpeg GIT, *and* that it affects more than just screen capture.

Easiest way to see it in action is to use one alsa and one video4linux input into some kind of stream muxer (which one, or which encoders are used doesn't seem to matter), then wait for a bit on the "Overwrite?" screen. If you wait on that screen, and the capture hardware doesn't have massive buffers available, you'll see an ALSA XRUN indicating the audio device was open and running the entire time the prompt was shown. Passing -y produces no XRUN.

My primary concern right now is that I've been trying to track down a subtle, constant A/V desync with properly timestamped ALSA/V4L buffers (hardware synced clocks), and I'm wondering if this issue is related. If pausing on the Overwrite prompt causes buffer overrun, I wonder if the small amount of code executed between the ALSA device opening and the V4L device opening with -y passed could still be introducing the slight (~20ms) constant desync.

I'm not giving a full command line because capture devices will vary between users, and a live source does seem to be required to trigger the issue (x11 screen grab, V4L capture, etc.).

Note that on my hardware, due to the tiny ALSA buffer sizes available, the A/V only goes out of sync by a little bit, with the primary symptom being silence in the encoded output for the duration of the Overwrite? prompt.

[grepfor]

Question for devs: This ticket was recently closed with "Reproduced by developer = 0". Has any attempt been made by the devs to reproduce it?

It seems to me that that given the information in the original ticket (that I filed in 2014) and the recent update by madscientist159 a few months ago, it should be 100% reproducible by the developers.

Can you recommend anything that I or madscientist159 can do to help resolve this? This bug is a major PITB, it seems to be affecting quite a few users, and has been in existence for at least six years.

I changed the status to "reopen" (not sure if that's allowed or not) with the hope that the devs will let me know what I can do to help get it resolved. Perhaps a better example, or whatever the case may be.

Thx.

[Carl Eugen Hoyos]

Please provide the command line you tested together with the complete, uncut console output to make this a valid ticket.

[grepfor]

The information you requested -- commandline and console output -- are shown in the first submission on this ticket, dated May 5, 2014 at 8:30:21 PM.

[Carl Eugen Hoyos]

Sorry, you misunderstand: You showed the console output for a version that was not supported at the time you opened the ticket (and is even less supported now). Please test current FFmpeg git head (the only version supported here) and provide the command line together with the complete, uncut console output.

[Carl Eugen Hoyos]

Replying to grepfor:

However, removing the -y option -- and assuming that a file named ofile.mkv already exists, so that ffmpeg asks whether to overwrite it -- results in skew between audio and video.

This (was and) is unavoidable.
What has always surprised me is that I don't see ffmpeg asking that in your console output...

[grepfor]

Ok, thanks.

If it's unavoidable though, it would make sense to document it as such under the "-y" option in the man page, and perhaps elsewhere. Imo, it is highly counterintuitive to most users that an option such as "-y" would result in unavoidable A/V skew.

As to why the "File exists: Overwrite?" query doesn't appear in the -report output... I have no explanation. I did not (at least not intentionally) make any edits to the file dumped by "-report". The -report option is presently documented in version "n4.2.1" as follows:

"-report Dump full command line and console output to a file named..."

[Carl Eugen Hoyos]

It only shows up on the console, not in the report file, I didn't know that as I rarely use the report option.

[grepfor]

If that's the case, then the doc for the "-report" option is incorrect. It specifically states that the "full command line and console output" is dumped to the report file.

Shall I file a bug report on that?

[Cigaes]

-report actually records the *log* output, which is almost the same as the *console* output, but not quite. The overwrite question is sent to stdio directly, not going through the log system.

[Cigaes]

Why are you using -report in interactive mode in the first place? -report is designed for cases when you cannot access the log output easily.

[grepfor]

Ok, then given what you say above, the man page doc for -report is indeed incorrect (or at best, misleading) as written.

Based on the present doc, users who are asked to file a ticket "including complete uncut console output" can expect that supplying the dumpfile obtained via "-report" will accomplish exactly that, yet -report evidently leaves out some key info that might be of help in diagnosing his issue.

Shall I file a ticket on the -report doc? Or is the eliding of stdio from the -report output unavoidable too? :)

[Cigaes]

The doc is already in the process of being updated. But you were asked the *console* output. Why did you give the -report output instead. The *console* output is a very accurate description of what you were asked.

[grepfor]

To answer your question as to why I used the "-report" output to provide the info for the bug report: The reason is straightforward: Because that option is documented as follows:

     -report Dump full command line and console output to a file named     
             program-YYYYMMDD-HHMMSS.log in the current directory.
             This file can be useful for bug reports.

So, the full line of reasoning is this: Since the bug reporting guidelines ask for the full commandline and console output, and since -report is documented as providing exactly that info, and is also documented as being "useful for bug reports"... I used it for the bug report. Seems pretty logical if y'ask me.