Concat demuxer is not closing file handles for old playlists

[Max Kossatz]

Summary of the bug:

on Ubuntu 18.04 (i guess on all linux distributions), when you reference a new playlist in a concat playlist file and this new playlist is getting played the file handle of the first playlist is not getting closed, even when the second playlist has the same name.

How to reproduce:

generate a test-video like this:

ffmpeg -f lavfi -i testsrc=s=720x576:r=12:d=4 -pix_fmt yuv422p -vf "drawbox=w=50:h=w:t=w:c=${c:?}" test.mkv

generate a playlist file called "test.txt" with this content:

ffconcat version 1.0
file 'test.mkv'
file 'test.txt'

invoke ffmpeg like this (streaming the file to a local udp port just to get it going forever):

ffmpeg -re -f concat -i test.txt -c copy -f rtp rtp://localhost:2000

Get the process number for ffmpeg (like ps xa|grep ffmpeg)
Check how many file handles this process takes:

lsof -p <processnumber>

You will see that over time the "test.txt" playlist file handle will get more and more, so the concat demuxer is not closing the file handle after reading the playlist. Same behaviour if you switch between different playlists or use pipes, even the bash example on ​https://trac.ffmpeg.org/wiki/Concatenate has this problem.
Over time this leads to cpu increase of the ffmpeg process until the machine runs out of file handles.

I tried it with different ffmpeg version from 4.1 to 4.2 (including on Ubuntu the version from the ppa:jonathonf/ffmpeg-4 repository) every time the same problem.

[Marton Balint]

What do you expect if you reference the playlist file in the playlist?

[Max Kossatz]

Well, the example references the same video over and over again and the file handle for the video is always getting closed every time when the video is finished, i would excpect the same for the playlist-file.

And, when you make 10 playlist-files and chain them together so that playlist 1 is referencing playlist 2 and so on the file handle for playlist 1, even hours after it was used (all videos processed from this play list) is not getting closed, but the file handles for all videos that ffmpeg processed from that playlist are closed.

[Marton Balint]

I guess you are referring to the "Automatically appending to the list file" example on the wiki, right?

The problem is that you are crearing an infinite recursion of playlists. That will eventually run out of resources, file descriptors first in your case.

FFmpeg only closes the playlist source file descriptor when that playlist finishes. None of the playlist files finishes, because the last entry is another playlist which never finishes, and so on.

If you need simple loop features, use -stream_loop, and not this endless playlist recursion, which is a *if it works it is not stupid* kind of hack and has its limitations.

[Max Kossatz]

I understand what you are saying, but i still think this is a bug or at least it should be mentioned somewhere.
Using playlists is the only way i found to dynamically feed new videos (by chaining playlists) to an existing, permanent running output-stream, -stream_loop is only for looping the same video.

[Marton Balint]

This may work for you as well depending on use case:

dummy.txt:

ffconcat version 1.0
file dummy.mkv
file dummy.mkv

ffmpeg -re -stream_loop -1 -i dummy.txt -f sdl2 none

You can seamlessly change the input by atomically replacing dummy.mkv.

The wiki should probably be updated that the chaining method has limitations.

[Max Kossatz]

Did not thought of this, nice idea, will try it!
Yes please, the wiki should be updated for this.

[Marton Balint]

I have updated the wiki, so closing this as wontfix.