filter_complex pipeline "segment;guided;concat" sometimes fails assertion

[breunigs]

To reproduce, run the following a couple of times:

ffmpeg -f lavfi -i testsrc2 -f lavfi -i testsrc2 \
  -filter_complex
    '[0]segment=timestamps=10[a][b];
     [b][1]guided=guidance=on[c];
     [a][c]concat=n=2' \
  -f matroska - | \
  head >/dev/null

It sometimes outputs

Assertion mainpic failed at libavfilter/framesync.c:401

instead of the expected

Conversion failed!

I'm not sure why it doesn't reproduce sometimes. Adding -threads 1 -filter_threads 1 -filter_complex_threads 1 to the CLI makes no difference, i.e. the bug can still be reproduced sometimes.

A full stacktrace for the reproducer looks like this:

#0  __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at ./nptl/pthread_kill.c:44
#1  0x00007f7d02c9debf in __pthread_kill_internal (threadid=<optimized out>, signo=6) at ./nptl/pthread_kill.c:78
#2  0x00007f7d02c49c82 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
#3  0x00007f7d02c324f0 in __GI_abort () at ./stdlib/abort.c:79
#4  0x00005568e8114f13 in ff_framesync_dualinput_get () at libavfilter/framesync.c:401
#5  0x00005568e81fe551 in process_frame () at libavfilter/vf_guided.c:319
#6  0x00005568e8114d46 in ff_framesync_activate () at libavfilter/framesync.c:364
#7  0x00005568e81feae5 in activate () at libavfilter/vf_guided.c:409
#8  0x00005568e80f6a89 in ff_filter_activate () at libavfilter/avfilter.c:1430
#9  0x00005568e80fc278 in ff_filter_graph_run_once () at libavfilter/avfiltergraph.c:1488
#10 0x00005568e80fdd76 in push_frame () at libavfilter/buffersrc.c:183
#11 0x00005568e80fe560 in av_buffersrc_close () at libavfilter/buffersrc.c:283
#12 0x00005568e80874f4 in send_eof () at fftools/ffmpeg_filter.c:2731
#13 0x00005568e8088443 in filter_thread () at fftools/ffmpeg_filter.c:3039
#14 0x00005568e80a18fd in task_wrapper () at fftools/ffmpeg_sched.c:2534
#15 0x00007f7d02c9c112 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:447
#16 0x00007f7d02d1a8f8 in __GI___clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78

captured from

ffmpeg version N-118231-g2f4ec16836 Copyright (c) 2000-2025 the FFmpeg developers
  built with gcc 14 (Debian 14.2.0-6)
  configuration: --enable-version3 --enable-gpl --enable-frei0r --enable-libx264 --disable-doc --disable-ffplay --disable-ffprobe --enable-debug=1 --disable-stripping --disable-optimizations --extra-cflags='-O0 -g' --extra-ldflags=-g
  libavutil      59. 54.101 / 59. 54.101
  libavcodec     61. 28.100 / 61. 28.100
  libavformat    61.  9.104 / 61.  9.104
  libavdevice    61.  4.100 / 61.  4.100
  libavfilter    10.  6.101 / 10.  6.101
  libswscale      8. 13.100 /  8. 13.100
  libswresample   5.  4.100 /  5.  4.100
  libpostproc    58.  4.100 / 58.  4.100

This fails for a Debian release version (i.e. not built by me) as well:

ffmpeg version 7.1-3 Copyright (c) 2000-2024 the FFmpeg developers
  built with gcc 14 (Debian 14.2.0-7)
  configuration: --prefix=/usr --extra-version=3 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --disable-libmfx --disable-omx --enable-gnutls --enable-libaom --enable-libass --enable-libbs2b --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libglslang --enable-libgme --enable-libgsm --enable-libharfbuzz --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-openal --enable-opencl --enable-opengl --disable-sndio --enable-libvpl --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-ladspa --enable-libbluray --enable-libcaca --enable-libdvdnav --enable-libdvdread --enable-libjack --enable-libpulse --enable-librabbitmq --enable-librist --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libx264 --enable-libzmq --enable-libzvbi --enable-lv2 --enable-sdl2 --enable-libplacebo --enable-librav1e --enable-pocketsphinx --enable-librsvg --enable-libjxl --enable-shared
  libavutil      59. 39.100 / 59. 39.100
  libavcodec     61. 19.100 / 61. 19.100
  libavformat    61.  7.100 / 61.  7.100
  libavdevice    61.  3.100 / 61.  3.100
  libavfilter    10.  4.100 / 10.  4.100
  libswscale      8.  3.100 /  8.  3.100
  libswresample   5.  3.100 /  5.  3.100
  libpostproc    58.  3.100 / 58.  3.100

• I checked as far back as bf0f996c1283731e56a7986a08002baa8ae5064c (~2021-09), which still has this issue.
• The issue goes away if not both in[0].before and in[0].after are set to EXT_INFINITY.
• I am able to reproduce this outside vf_guided using another (not upstreamed) filter, so it doesn't seem to be strictly related to vf_guided. It's just the only filter that sets before=after=EXT_INFINITY.
• shorter timestamps for vf_segment seem to fix the issue, or at least make it very hard to reproduce. I was able to break it with values as low as segment=timestamps=4.99 but not below.
• replacing vf_segment with e.g. vf_split "fixes" the issue. So does removing the concat filter (and writing both outputs to files or stdout)
• adding a -frames:v 100 after the filters also fixes the issue. The number of frames needs to reasonably small, e.g. increasing the frames to 150 already shows the issue for me.