#8450 closed defect (invalid)
x11grab doesn't work with headless X servers
| Reported by: | eero-t | Owned by: | |
|---|---|---|---|
| Priority: | normal | Component: | avdevice |
| Version: | git-master | Keywords: | |
| Cc: | Marton Balint | Blocked By: | |
| Blocking: | Reproduced by developer: | no | |
| Analyzed by developer: | no |
Description
Summary of the bug:
FFmpeg x11grab doesn't work with headless X server configurations.
How to reproduce:
- Start headless X server:
Xvfb :99 -screen 0 1920x1080x24 -nolisten tcp
- Try streaming that with FFmepg:
$ ffmpeg -an -f x11grab -video_size 1920x1080 -framerate 2 -i :99 -c:v libx264 -tune zerolatency -pix_fmt nv12 -b:v 5000k -f flv rtmp://hostname/live/desktop ffmpeg version 4.1.4-1build2 Copyright (c) 2000-2019 the FFmpeg developers built with gcc 9 (Ubuntu 9.2.1-4ubuntu1) ... [x11grab @ 0x5652189c7b80] Cannot get the image data event_error: response_type:0 error_code:128 sequence:9 resource_id:4194304 minor_code:4 major_code:130. [x11grab @ 0x5652189c7b80] Stream #0: not enough frames to estimate rate; consider increasing probesize Input #0, x11grab, from ':99': Duration: N/A, bitrate: N/A Stream #0:0: Video: rawvideo (BGR[0] / 0x524742), bgr0, 1920x1080, 2 fps, 1000k tbr, 1000k tbn, 1000k tbc Stream mapping: Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (libx264)) ... Output #0, flv, to 'rtmp://192.168.0.8/live/desktop': Metadata: encoder : Lavf58.20.100 Stream #0:0: Video: h264 (libx264) ([7][0][0][0] / 0x0007), nv12, 1920x1080, q=-1--1, 5000 kb/s, 2 fps, 1k tbn, 8 tbc Metadata: encoder : Lavc58.35.100 libx264 Side data: cpb: bitrate max/min/avg: 0/0/5000000 buffer size: 0 vbv_delay: -1 frame= 2 fps=0.0 q=7.0 size= 2kB time=00:00:00.12 bitrate= 104.4kbits/s dup=0 drop=4 speed=0.2frame= 2 fps=1.8 q=7.0 size= 2kB time=00:00:00.12 bitrate= 104.4kbits/s dup=0 drop=9 speed=0.1frame= 2 fps=1.1 q=7.0 size= 2kB time=00:00:00.12 bitrate= 104.4kbits/s dup=0 drop=14 speed=0.frame= 2 fps=0.9 q=7.0 size= 2kB time=00:00:00.12 bitrate= 104.4kbits/s dup=0 drop=18 speed=0. ...
Outcome is that FFmpeg is forever stuck at dropping frames.
FFmpeg code outputs above warning from two places here:
https://github.com/FFmpeg/FFmpeg/blob/master/libavdevice/xcbgrab.c
Checking the X protocol headers, tells that "minor_code:4 major_code:130" is ShmGetImage(), i.e. culprit is xcbgrab_frame_shm() function.
In the Xvfb case, I can add "-shmem" option to get X root window to be available as shared memory:
https://www.x.org/archive/X11R7.5/doc/man/man1/Xvfb.1.html
HOWEVER, while that gets FFmpeg streaming working, then the use-cases which output I'd like to stream, don't work! (Those use-cases use GL with Mesa)
If I try to use Xdummy instead of Xvfb (to get 3D acceleration working for X clients):
FFmpeg fails with that too, but I didn't find any Xdummy option that would allow FFmpeg to work with it.
In summary, there are several issues:
- Error message that's incomprehensible to normal users
- Not stopping when things have failed completely (=FFmpeg can't acquire any input frames)
- FFmpeg code seems to have also a non-SHM function for getting the frames, but FFmpeg doesn't seem to fallback to that, or that part is broken for some other reason which doesn't show up in FFmpeg output
PS. "xdpyinfo" shows following for Xdummy:
name of display: :99
version number: 11.0
vendor string: The X.Org Foundation
vendor release number: 12005000
X.Org version: 1.20.5
maximum request size: 16777212 bytes
motion buffer size: 256
bitmap unit, bit order, padding: 32, LSBFirst, 32
image byte order: LSBFirst
number of supported pixmap formats: 7
supported pixmap formats:
depth 1, bits_per_pixel 1, scanline_pad 32
depth 4, bits_per_pixel 8, scanline_pad 32
depth 8, bits_per_pixel 8, scanline_pad 32
depth 15, bits_per_pixel 16, scanline_pad 32
depth 16, bits_per_pixel 16, scanline_pad 32
depth 24, bits_per_pixel 32, scanline_pad 32
depth 32, bits_per_pixel 32, scanline_pad 32
keycode range: minimum 8, maximum 255
focus: PointerRoot
number of extensions: 28
BIG-REQUESTS
Composite
DAMAGE
DOUBLE-BUFFER
DPMS
DRI2
DRI3
GLX
Generic Event Extension
MIT-SCREEN-SAVER
MIT-SHM
Present
RANDR
RECORD
RENDER
SECURITY
SHAPE
SYNC
X-Resource
XC-MISC
XFIXES
XFree86-DGA
XFree86-VidModeExtension
XINERAMA
XInputExtension
XKEYBOARD
XTEST
XVideo
default screen number: 0
number of screens: 1
screen #0:
dimensions: 1920x1080 pixels (508x285 millimeters)
resolution: 96x96 dots per inch
depths (7): 24, 1, 4, 8, 15, 16, 32
root window id: 0x14a
depth of root window: 24 planes
number of colormaps: minimum 1, maximum 1
default colormap: 0x20
default number of colormap cells: 256
preallocated pixels: black 0, white 16777215
options: backing-store WHEN MAPPED, save-unders NO
...
I.e. MIT-SHM extension is suppported, X root window just isn't using it.
Change History (11)
comment:1 by , 5 years ago
| Component: | avdevice → undetermined |
|---|---|
| Resolution: | → needs_more_info |
| Status: | new → closed |
| Version: | 4.1.4 → unspecified |
comment:2 by , 5 years ago
| Component: | undetermined → avdevice |
|---|---|
| Resolution: | needs_more_info |
| Status: | closed → reopened |
| Version: | unspecified → git-master |
comment:3 by , 5 years ago
In the Xvfb case, I can add "-shmem" option to get X root window to be available as shared memory:
HOWEVER, while that gets FFmpeg streaming working, then the use-cases which output I'd like to stream, don't work!
FYI: with Xvfb "-shmem" root window option, ShmPutImage() calls will fail in (fullscreen) applications.
comment:4 by , 5 years ago
| Cc: | added |
|---|
Well, you should try the latest git master because there were some issues with XCB grab timestamps for the last few weeks. Also make sure you can reproduce this with local (non-networked) output and without hwaccel.
comment:5 by , 5 years ago
Well, you should try the latest git master because there were some issues with XCB grab timestamps for the last few weeks. Also make sure you can reproduce this with local (non-networked) output and without hwaccel.
With git-master from last evening, I'll see:
$ ffmpeg -hwaccel vaapi -vaapi_device /dev/dri/renderD128 -an -f x11grab -video_size 1920x1080 -framerate 2 -i :99 -c:v h264_vaapi -vf format=nv12,hwupload -b:v 5000k -f flv rtmp://192.168.0.8/live/desktop
ffmpeg version N-96204-gf651b18c19 Copyright (c) 2000-2019 the FFmpeg developers
built with gcc 7 (Ubuntu 7.4.0-1ubuntu1~18.04.1)
configuration: --prefix=/opt/ --enable-libmfx --enable-vaapi --enable-sdl2 --disable-libx265 --disable-libx264 --disable-libvpx --enable-libvorbis --enable-libopus --disable-libmp3lame --disable-libass --disable-sndio --enable-libfreetype --enable-gpl --disable-doc
libavutil 56. 38.100 / 56. 38.100
libavcodec 58. 65.100 / 58. 65.100
libavformat 58. 35.101 / 58. 35.101
libavdevice 58. 9.101 / 58. 9.101
libavfilter 7. 70.101 / 7. 70.101
libswscale 5. 6.100 / 5. 6.100
libswresample 3. 6.100 / 3. 6.100
libpostproc 55. 6.100 / 55. 6.100
[x11grab @ 0x556499d1fa80] Cannot get the image data event_error: response_type:0 error_code:128 sequence:9 resource_id:4194304 minor_code:4 major_code:130.
[x11grab @ 0x556499d1fa80] Continuing without shared memory.
[x11grab @ 0x556499d1fa80] Stream #0: not enough frames to estimate rate; consider increasing probesize
Input #0, x11grab, from ':99':
Duration: N/A, bitrate: N/A
Stream #0:0: Video: rawvideo (BGR[0] / 0x524742), bgr0, 1920x1080, 2 fps, 1000k tbr, 1000k tbn, 1000k tbc
Stream mapping:
Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (h264_vaapi))
Press [q] to stop, [?] for help
[graph 0 input from stream 0:0 @ 0x556499d43d00] sws_param option is deprecated and ignored
Output #0, flv, to 'rtmp://192.168.0.8/live/desktop':
Metadata:
encoder : Lavf58.35.101
Stream #0:0: Video: h264 (h264_vaapi) (High) ([7][0][0][0] / 0x0007), vaapi_vld, 1920x1080, q=-1--1, 5000 kb/s, 2 fps, 1k tbn, 2 tbc
Metadata:
encoder : Lavc58.65.100 h264_vaapi
[flv @ 0x556499d29c40] Failed to update header with correct duration.N/A dup=0 drop=63 speed=N/A
[flv @ 0x556499d29c40] Failed to update header with correct filesize.
frame= 2 fps=0.1 q=-0.0 Lsize= 1kB time=00:00:00.00 bitrate=7848.0kbits/s dup=0 drop=64 speed=3.09e-05x
Same issue without HW acceleration to local file:
$ ffmpeg -y -an -f x11grab -video_size 1920x1080 -framerate 2 -i :99 out.mp4
ffmpeg version N-96204-gf651b18c19 Copyright (c) 2000-2019 the FFmpeg developers
built with gcc 7 (Ubuntu 7.4.0-1ubuntu1~18.04.1)
configuration: --prefix=/opt/ --enable-libmfx --enable-vaapi --enable-sdl2 --disable-libx265 --disable-libx264 --disable-libvpx --enable-libvorbis --enable-libopus --disable-libmp3lame --disable-libass --disable-sndio --enable-libfreetype --enable-gpl --disable-doc
libavutil 56. 38.100 / 56. 38.100
libavcodec 58. 65.100 / 58. 65.100
libavformat 58. 35.101 / 58. 35.101
libavdevice 58. 9.101 / 58. 9.101
libavfilter 7. 70.101 / 7. 70.101
libswscale 5. 6.100 / 5. 6.100
libswresample 3. 6.100 / 3. 6.100
libpostproc 55. 6.100 / 55. 6.100
[x11grab @ 0x55a4dccf4380] Cannot get the image data event_error: response_type:0 error_code:128 sequence:9 resource_id:4194304 minor_code:4 major_code:130.
[x11grab @ 0x55a4dccf4380] Continuing without shared memory.
[x11grab @ 0x55a4dccf4380] Stream #0: not enough frames to estimate rate; consider increasing probesize
Input #0, x11grab, from ':99':
Duration: N/A, bitrate: N/A
Stream #0:0: Video: rawvideo (BGR[0] / 0x524742), bgr0, 1920x1080, 2 fps, 1000k tbr, 1000k tbn, 1000k tbc
Stream mapping:
Stream #0:0 -> #0:0 (rawvideo (native) -> mpeg4 (native))
Press [q] to stop, [?] for help
[graph 0 input from stream 0:0 @ 0x55a4dcd07d80] sws_param option is deprecated and ignored
Output #0, mp4, to 'out.mp4':
Metadata:
encoder : Lavf58.35.101
Stream #0:0: Video: mpeg4 (mp4v / 0x7634706D), yuv420p, 1920x1080, q=2-31, 200 kb/s, 2 fps, 16384 tbn, 2 tbc
Metadata:
encoder : Lavc58.65.100 mpeg4
Side data:
cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: N/A
frame= 3 fps=0.0 q=2.0 size= 0kB time=00:00:01.00 bitrate= 0.4kbits/s dup=0 drop=3 speed=1.83xframe= 3 fps=1.9 q=2.0 size= 0kB time=00:00:01.00 bitrate= 0.4kbits/s dup=0 drop=5 speed=0.647frame= 3 fps=1.5 q=2.0 size= 0kB time=00:00:01.00 bitrate= 0.4kbits/s dup=0 drop=6 speed=0.489frame= 3 fps=1.0 q=2.0 size= 0kB time=00:00:01.00 bitrate= 0.4kbits/s dup=0 drop=8 speed=0.328frame= 3 fps=0.7 q=2.0 size= 0kB time=00:00:01.00 bitrate= 0.4kbits/s dup=0 drop=10 speed=0.24frame= 3 fps=0.7 q=2.0 size= 0kB time=00:00:01.00 bitrate= 0.4kbits/s dup=0 drop=11 speed=0.22frame= 3 fps=0.5 q=2.0 size= 0kB time=00:00:01.00 bitrate= 0.4kbits/s dup=0 drop=13 speed=0.18frame= 3 fps=0.5 q=2.0 size= 0kB time=00:00:01.00 bitrate= 0.4kbits/s dup=0 drop=15 speed=0.15frame= 3 fps=0.4 q=2.0 size= 0kB time=00:00:01.00 bitrate= 0.4kbits/s dup=0 drop=16 speed=0.14frame= 3 fps=0.4 q=2.0 size= 0kB time=00:00:01.00 bitrate= 0.4kbits/s dup=0 drop=18 speed=0.12frame= 3 fps=0.3 q=2.0 size= 0kB time=00:00:01.00 bitrate= 0.4kbits/s dup=0 drop=20 speed=0.11frame= 3 fps=0.3 q=2.0 size= 0kB time=00:00:01.00 bitrate= 0.4kbits/s dup=0 drop=22 speed=0.09frame= 3 fps=0.3 q=2.0 size= 0kB time=00:00:01.00 bitrate= 0.4kbits/s dup=0 drop=23 speed=0.09frame= 3 fps=0.3 q=2.0 size= 0kB time=00:00:01.00 bitrate= 0.4kbits/s dup=0 drop=24 speed=0.09frame= 3 fps=0.2 q=2.0 size= 0kB time=00:00:01.00 bitrate= 0.4kbits/s dup=0 drop=26 speed=0.08frame= 3 fps=0.2 q=2.0 Lsize= 25kB time=00:00:01.00 bitrate= 203.6kbits/s dup=0 drop=27 speed=0.0797x
Now Fmpeg says that it's continuing without shared memory, but all frames are still dropped.
What's worse, last night git-master version says that (and fails), even when Xvfb has been started with -shmem option.
comment:6 by , 5 years ago
Maybe you are using some nightly builds, but you have not tried the latest git master, because the change I was referring to was committed after f651b18c19 (the version which you were using). Please try the latest...
comment:7 by , 5 years ago
Ok, now it isn't anymore dropping frames with Xvfb:
$ Xvfb :99 -screen 0 1920x1080x24 -nolisten tcp &
$ ffmpeg -y -an -f x11grab -video_size 1920x1080 -framerate 8 -i :99 out.mp4
ffmpeg version N-96235-g3670a10c2d Copyright (c) 2000-2020 the FFmpeg developers
built with gcc 7 (Ubuntu 7.4.0-1ubuntu1~18.04.1)
configuration: --prefix=/opt --enable-libmfx --enable-vaapi --enable-sdl2 --disable-libx265 --disable-libx264 --disable-libvpx --enable-libvorbis --enable-libopus --disable-libmp3lame --disable-libass --disable-sndio --enable-libfreetype --enable-gpl --disable-doc
libavutil 56. 38.100 / 56. 38.100
libavcodec 58. 65.102 / 58. 65.102
libavformat 58. 35.101 / 58. 35.101
libavdevice 58. 9.102 / 58. 9.102
libavfilter 7. 70.101 / 7. 70.101
libswscale 5. 6.100 / 5. 6.100
libswresample 3. 6.100 / 3. 6.100
libpostproc 55. 6.100 / 55. 6.100
[x11grab @ 0x564d93f4b3c0] Cannot get the image data event_error: response_type:0 error_code:128 sequence:9 resource_id:2097152 minor_code:4 major_code:130.
[x11grab @ 0x564d93f4b3c0] Continuing without shared memory.
[x11grab @ 0x564d93f4b3c0] Stream #0: not enough frames to estimate rate; consider increasing probesize
Input #0, x11grab, from ':99':
Duration: N/A, start: 1577985187.298908, bitrate: N/A
Stream #0:0: Video: rawvideo (BGR[0] / 0x524742), bgr0, 1920x1080, 8 fps, 1000k tbr, 1000k tbn, 1000k tbc
Stream mapping:
Stream #0:0 -> #0:0 (rawvideo (native) -> mpeg4 (native))
Press [q] to stop, [?] for help
Output #0, mp4, to 'out.mp4':
Metadata:
encoder : Lavf58.35.101
Stream #0:0: Video: mpeg4 (mp4v / 0x7634706D), yuv420p, 1920x1080, q=2-31, 200 kb/s, 8 fps, 16384 tbn, 8 tbc
Metadata:
encoder : Lavc58.65.102 mpeg4
Side data:
cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: N/A
frame= 192 fps=8.0 q=2.0 Lsize= 543kB time=00:00:23.87 bitrate= 186.2kbits/s speed= 1x
I'll check it more tomorrow (Xdummy, the actual output etc).
comment:8 by , 5 years ago
| Resolution: | → fixed |
|---|---|
| Status: | reopened → closed |
Reopen if you can reproduce the original issue with the git master.
comment:9 by , 5 years ago
| Resolution: | fixed → invalid |
|---|
comment:10 by , 5 years ago
| Resolution: | invalid → fixed |
|---|
Whoever is running the "cehoyos" bot, please fix it, it's randomly messing up bug statuses. Correcting it again...
comment:11 by , 5 years ago
| Resolution: | fixed → invalid |
|---|
Note:
See TracTickets
for help on using tickets.
cehoyos, when you close bug as needing more info, indicate what information is missing, and don't remove already given information.
In case the problem was testing a release, I tried now also git build of FFmpeg:
ffmpeg -hwaccel vaapi -vaapi_device /dev/dri/renderD128 -an -f x11grab -video_size 1920x1080 -framerate 2 -i :99 -c:v h264_vaapi -vf format=nv12,hwupload -b:v 5000k -f flv rtmp://192.168.0.8/live/desktop_1 ffmpeg version N-95996-ge3dddf2142 Copyright (c) 2000-2019 the FFmpeg developers built with gcc 7 (Ubuntu 7.4.0-1ubuntu1~18.04.1) configuration: --prefix=/opt/ --enable-libmfx --enable-vaapi --enable-sdl2 --disable-libx265 --disable-libx264 --disable-libvpx --enable-libvorbis --enable-libopus --disable-libmp3lame --disable-libass --disable-sndio --enable-libfreetype --enable-gpl --disable-doc libavutil 56. 36.101 / 56. 36.101 libavcodec 58. 64.101 / 58. 64.101 libavformat 58. 35.101 / 58. 35.101 libavdevice 58. 9.101 / 58. 9.101 libavfilter 7. 68.100 / 7. 68.100 libswscale 5. 6.100 / 5. 6.100 libswresample 3. 6.100 / 3. 6.100 libpostproc 55. 6.100 / 55. 6.100 [x11grab @ 0x55bf40067a40] Cannot get the image data event_error: response_type:0 error_code:128 sequence:9 resource_id:4194304 minor_code:4 major_code:130. [x11grab @ 0x55bf40067a40] Stream #0: not enough frames to estimate rate; consider increasing probesize Input #0, x11grab, from ':99': Duration: N/A, bitrate: N/A Stream #0:0: Video: rawvideo (BGR[0] / 0x524742), bgr0, 1920x1080, 2 fps, 1000k tbr, 1000k tbn, 1000k tbc Stream mapping: Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (h264_vaapi)) Press [q] to stop, [?] for help Output #0, flv, to 'rtmp://192.168.0.8/live/desktop_1': Metadata: encoder : Lavf58.35.101 Stream #0:0: Video: h264 (h264_vaapi) (High) ([7][0][0][0] / 0x0007), vaapi_vld, 1920x1080, q=-1--1, 5000 kb/s, 2 fps, 1k tbn, 2 tbc Metadata: encoder : Lavc58.64.101 h264_vaapi frame= 2 fps=0.0 q=-0.0 size= 1kB time=-00:00:00.49 bitrate=N/A dup=0 drop=734 speed=N/AAnd as can be seen from above, that has exactly the same problem, all frames are "dropped" (because FFmpeg x11grab fails to acquire them).