audio stream start_time & duration overspan when encoding & split a video (-ss, -t)

[ronny1982]

I'm trying to extract and encode a segment of a video with a given start time and duration.

Until version 0.10.7 the start time and duration of the extracted audio stream were almost accurate compared to the video stream.

Since version 0.11.3 additional audio packets are appended that lies outside of the requested duration -> the audio stream is 'much' longer than the video stream compared to the result with version 0.10.7.

Since version 1.1.4 the behavior changed again, the audio packets no longer starts with pts_time 0.0, instead they got an offset. This offset will also be added to the regular duration of the last audio packet. Technically the values belongs to the requested timespan:
first_packet_pts_time + packet_duration > -ss <start_time>
last packet_pts_time < -ss <start_time> + -t <duration>
However the output of version 0.10.7 is more logical...

Why making a fuss about 20~50ms additional difference between the stream durations? The higher accuracy (like in version 0.10.7) is required when you want to concatenate multiple segments, where different stream start times & durations playing an important role to prevent 'gaps' in the video stream (-> ​explained here)

---

shell script for testing:

#!/bin/sh

cd "$(dirname "$0")"

# extract and encode segments with different ffmpeg versions
ffmpeg-0.10.7 -i "Agnis Philosophy.mp4" -strict experimental -f mp4 -c:a aac -c:v mpeg4 -ss 00:00:11.000 -t 00:00:12.666 -y "agnis-part_0.10.7.mp4"
ffmpeg-0.11.3 -i "Agnis Philosophy.mp4" -strict experimental -f mp4 -c:a aac -c:v mpeg4 -ss 00:00:11.000 -t 00:00:12.666 -y "agnis-part_0.11.3.mp4"
ffmpeg-1.0.6 -i "Agnis Philosophy.mp4" -strict experimental -f mp4 -c:a aac -c:v mpeg4 -ss 00:00:11.000 -t 00:00:12.666 -y "agnis-part_1.0.6.mp4"
ffmpeg-1.1.4 -i "Agnis Philosophy.mp4" -strict experimental -f mp4 -c:a aac -c:v mpeg4 -ss 00:00:11.000 -t 00:00:12.666 -y "agnis-part_1.1.4.mp4"
ffmpeg-1.2 -i "Agnis Philosophy.mp4" -strict experimental -f mp4 -c:a aac -c:v mpeg4 -ss 00:00:11.000 -t 00:00:12.666 -y "agnis-part_1.2.mp4"

# extract stream infos from each encoded segment
ffprobe -show_streams 'agnis-part_0.10.7.mp4' | grep 'STREAM\|start_time\|duration\|nb_frames' > 'info_0.10.7.txt'
ffprobe -show_streams 'agnis-part_0.11.3.mp4' | grep 'STREAM\|start_time\|duration\|nb_frames' > 'info_0.11.3.txt'
ffprobe -show_streams 'agnis-part_1.0.6.mp4' | grep 'STREAM\|start_time\|duration\|nb_frames' > 'info_1.0.6.txt'
ffprobe -show_streams 'agnis-part_1.1.4.mp4' | grep 'STREAM\|start_time\|duration\|nb_frames' > 'info_1.1.4.txt'
ffprobe -show_streams 'agnis-part_1.2.mp4' | grep 'STREAM\|start_time\|duration\|nb_frames' > 'info_1.2.txt'

# extract packet infos from each encoded segment
ffprobe -show_packets 'agnis-part_0.10.7.mp4' | grep 'PACKET\|codec_type\|pts_time\|dts_time\|duration_time' > 'packets_0.10.7.txt'
ffprobe -show_packets 'agnis-part_0.11.3.mp4' | grep 'PACKET\|codec_type\|pts_time\|dts_time\|duration_time' > 'packets_0.11.3.txt'
ffprobe -show_packets 'agnis-part_1.0.6.mp4' | grep 'PACKET\|codec_type\|pts_time\|dts_time\|duration_time' > 'packets_1.0.6.txt'
ffprobe -show_packets 'agnis-part_1.1.4.mp4' | grep 'PACKET\|codec_type\|pts_time\|dts_time\|duration_time' > 'packets_1.1.4.txt'
ffprobe -show_packets 'agnis-part_1.2.mp4' | grep 'PACKET\|codec_type\|pts_time\|dts_time\|duration_time' > 'packets_1.2.txt'

---

Taking a look into some selected stream infos of each encoded segment from ffprobe. You can notice the 'evolution' of the audio stream (first stream is video, second stream is audio):

ffmpeg-0.9.2 & ffmpeg-0.10.7	ffmpeg-0.11.3 & ffmpeg-1.0.6	fmpeg-1.1.4 & ffmpeg-1.2
[STREAM]	[STREAM]	[STREAM]
start_time=0.000000	start_time=0.000000	start_time=0.000000
duration=12.666667	duration=12.666667	duration=12.666667
nb_frames=380	nb_frames=380	nb_frames=380
[/STREAM]	[/STREAM]	[/STREAM]
[STREAM]	[STREAM]	[STREAM]
start_time=0.000000	start_time=0.000000	start_time=-0.016961
duration=12.678095	duration=12.701315	duration=12.701315
nb_frames=546	nb_frames=547	nb_frames=547
[/STREAM]	[/STREAM]	[/STREAM]

---

Source Video:

• ​Agnis Philosophy.mp4 (134 MB)

Results:

• ​Segment 0.10.7 (3.5 MB)
• ​Segment 0.11.3 (3.5 MB)
• ​Segment 1.0.6 (3.5 MB)
• ​Segment 1.1.4 (3.5 MB)
• ​Segment 1.2 (3.5 MB)
• ​Stream & Packet Infos tar.gz (34 KB)

*_{not sure if this behavior is a defect or is wanted by the developers, so setting the ticket to: enhancement with minor priority}

[ronny1982]

Additional Notes:

• the timing difference depends on audio codec
• when using audio copy all ffmpeg versions works like expected (only packets with timestamps in the requested timespan where copied)