Unplayable mpeg ts after converting mp3 to mp4 with image and then converting mp4 to mpeg ts

[tmeisenh]

I am having issues converting an mp3 to an mp4 with a single watermark image and then converting that mp4 into mpeg ts for streaming.

I have a folder out on dropbox containing the mp3 as well as a shell script to demonstrate the problem as well as my debugging. You can get it here: ​https://www.dropbox.com/sh/r4zc39y68gwzt7f/AAD6X2i_JbAU6nASeXBt90mXa?dl=0

Note: all of the below commands are in the test_bug.sh

What I have observed is that if I convert the mp3 to mp4 with single watermark image then the resulting mp4 has a duration that is a little longer than the original mp3.
If I then try and convert this mp4 to mpeg ts then the last segment will not have an audio stream. ffprobe verifies this.

ffmpeg -i mack_the_knife.mp3 -loop 1 -i watermark.jpg -pix_fmt yuv420p -shortest out.mp4
ffmpeg version 2.8.4 Copyright (c) 2000-2015 the FFmpeg developers

built with Apple LLVM version 7.0.2 (clang-700.1.81)
configuration: --prefix=/usr/local/Cellar/ffmpeg/2.8.4 --enable-shared --enable-pthreads --enable-gpl --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-opencl --enable-libx264 --enable-libmp3lame --enable-libvo-aacenc --enable-libxvid --enable-vda
libavutil 54. 31.100 / 54. 31.100
libavcodec 56. 60.100 / 56. 60.100
libavformat 56. 40.101 / 56. 40.101
libavdevice 56. 4.100 / 56. 4.100
libavfilter 5. 40.101 / 5. 40.101
libavresample 2. 1. 0 / 2. 1. 0
libswscale 3. 1.101 / 3. 1.101
libswresample 1. 2.101 / 1. 2.101
libpostproc 53. 3.100 / 53. 3.100

[mp3 @ 0x7ff6f9800000] Skipping 0 bytes of junk at 292.
[mp3 @ 0x7ff6f9800000] Estimating duration from bitrate, this may be inaccurate
Input #0, mp3, from 'mack_the_knife.mp3':

Metadata:

title : Track 7
album : Commercial DEMO
artist : Jon Novi
genre : Other
track : 7/17
encoder : Audiograbber 1.83.01, Fraunhofer acm, 56 kBit/s, 22,050 Hz, Stereo, HQ = off

Duration: 00:01:48.13, start: 0.000000, bitrate: 56 kb/s

Stream #0:0: Audio: mp3, 22050 Hz, stereo, s16p, 56 kb/s

[mjpeg @ 0x7ff6fa000c00] Changeing bps to 8
Input #1, image2, from 'watermark.jpg':

Duration: 00:00:00.04, start: 0.000000, bitrate: 11947 kb/s

Stream #1:0: Video: mjpeg, yuvj444p(pc, bt470bg/unknown/unknown), 768x768, 25 fps, 25 tbr, 25 tbn, 25 tbc

[swscaler @ 0x7ff6f98bb800] deprecated pixel format used, make sure you did set range correctly
[libx264 @ 0x7ff6f98b9a00] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
[libx264 @ 0x7ff6f98b9a00] profile High, level 3.1
[libx264 @ 0x7ff6f98b9a00] 264 - core 148 r2601 a0cd7d3 - H.264/MPEG-4 AVC codec - Copyleft 2003-2015 - ​http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=6 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to 'out.mp4':

Metadata:

title : Track 7
album : Commercial DEMO
artist : Jon Novi
genre : Other
track : 7/17
encoder : Lavf56.40.101
Stream #0:0: Video: h264 (libx264) ([33][0][0][0] / 0x0021), yuv420p, 768x768, q=-1--1, 25 fps, 12800 tbn, 25 tbc
Metadata:

encoder : Lavc56.60.100 libx264

Stream #0:1: Audio: aac (libvo_aacenc) ([64][0][0][0] / 0x0040), 22050 Hz, stereo, s16, 128 kb/s
Metadata:

encoder : Lavc56.60.100 libvo_aacenc

Stream mapping:

Stream #1:0 -> #0:0 (mjpeg (native) -> h264 (libx264))
Stream #0:0 -> #0:1 (mp3 (native) -> aac (libvo_aacenc))

Press [q] to stop, ? for help
frame= 2767 fps=192 q=-1.0 Lsize= 1920kB time=00:01:50.60 bitrate= 142.2kbits/s
video:152kB audio:1699kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 3.787704%
[libx264 @ 0x7ff6f98b9a00] frame I:12 Avg QP: 7.92 size: 5646
[libx264 @ 0x7ff6f98b9a00] frame P:697 Avg QP: 9.25 size: 39
[libx264 @ 0x7ff6f98b9a00] frame B:2058 Avg QP:12.67 size: 29
[libx264 @ 0x7ff6f98b9a00] consecutive B-frames: 0.8% 0.0% 0.0% 99.2%
[libx264 @ 0x7ff6f98b9a00] mb I I16..4: 65.4% 28.6% 6.0%
[libx264 @ 0x7ff6f98b9a00] mb P I16..4: 0.0% 0.0% 0.0% P16..4: 0.0% 0.0% 0.0% 0.0% 0.0% skip:100.0%
[libx264 @ 0x7ff6f98b9a00] mb B I16..4: 0.0% 0.0% 0.0% B16..8: 0.0% 0.0% 0.0% direct: 0.0% skip:100.0% L0:34.1% L1:65.9% BI: 0.0%
[libx264 @ 0x7ff6f98b9a00] 8x8 transform intra:28.6% inter:0.5%
[libx264 @ 0x7ff6f98b9a00] coded y,uvDC,uvAC intra: 3.6% 0.0% 0.0% inter: 0.0% 0.0% 0.0%
[libx264 @ 0x7ff6f98b9a00] i16 v,h,dc,p: 96% 0% 3% 0%
[libx264 @ 0x7ff6f98b9a00] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 31% 27% 42% 0% 0% 0% 0% 0% 0%
[libx264 @ 0x7ff6f98b9a00] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 37% 14% 21% 7% 4% 5% 3% 4% 3%
[libx264 @ 0x7ff6f98b9a00] i8c dc,h,v,p: 100% 0% 0% 0%
[libx264 @ 0x7ff6f98b9a00] Weighted P-Frames: Y:0.0% UV:0.0%
[libx264 @ 0x7ff6f98b9a00] ref P L0: 99.0% 0.0% 1.0%
[libx264 @ 0x7ff6f98b9a00] ref B L0: 44.5% 55.5%
[libx264 @ 0x7ff6f98b9a00] kb/s:11.17

## convert the mp4 to mpegts

ffmpeg -i out.mp4 -vcodec libx264 -bsf:v h264_mp4toannexb -strict strict -flags -global_header -dcodec copy -acodec copy -q:v 0 -map 0 -f segment -segment_list play.m3u8 -segment_time 10 -segment_list_flags +live -reset_timestamps 1 -segment_list_entry_prefix '​http://localhost:8000/' ./%06d.ts
ffmpeg version 2.8.4 Copyright (c) 2000-2015 the FFmpeg developers

built with Apple LLVM version 7.0.2 (clang-700.1.81)
configuration: --prefix=/usr/local/Cellar/ffmpeg/2.8.4 --enable-shared --enable-pthreads --enable-gpl --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-opencl --enable-libx264 --enable-libmp3lame --enable-libvo-aacenc --enable-libxvid --enable-vda
libavutil 54. 31.100 / 54. 31.100
libavcodec 56. 60.100 / 56. 60.100
libavformat 56. 40.101 / 56. 40.101
libavdevice 56. 4.100 / 56. 4.100
libavfilter 5. 40.101 / 5. 40.101
libavresample 2. 1. 0 / 2. 1. 0
libswscale 3. 1.101 / 3. 1.101
libswresample 1. 2.101 / 1. 2.101
libpostproc 53. 3.100 / 53. 3.100

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'out.mp4':

Metadata:

major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
title : Track 7
artist : Jon Novi
album : Commercial DEMO
encoder : Lavf56.40.101
genre : Other
track : 7/17

Duration: 00:01:50.68, start: 0.072562, bitrate: 142 kb/s

Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 768x768, 11 kb/s, 25 fps, 25 tbr, 12800 tbn, 50 tbc (default)
Metadata:

handler_name : VideoHandler

Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 22050 Hz, stereo, fltp, 127 kb/s (default)
Metadata:

handler_name : SoundHandler

[libx264 @ 0x7ff19d032200] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
[libx264 @ 0x7ff19d032200] profile High, level 3.1
[segment @ 0x7ff19d031000] Codec for stream 0 does not use global headers but container format requires global headers
[segment @ 0x7ff19d031000] Codec for stream 1 does not use global headers but container format requires global headers
Output #0, segment, to './%06d.ts':

Metadata:

major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
title : Track 7
artist : Jon Novi
album : Commercial DEMO
track : 7/17
genre : Other
encoder : Lavf56.40.101
Stream #0:0(und): Video: h264 (libx264), yuv420p, 768x768, q=-1--1, 25 fps, 90k tbn, 25 tbc (default)
Metadata:

handler_name : VideoHandler
encoder : Lavc56.60.100 libx264

Stream #0:1(und): Audio: aac (mp4a / 0x6134706D), 22050 Hz, stereo, 127 kb/s (default)
Metadata:

handler_name : SoundHandler

Stream mapping:

Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
Stream #0:1 -> #0:1 (copy)

Press [q] to stop, ? for help
frame= 2769 fps=420 q=-1.0 Lsize=N/A time=00:01:50.68 bitrate=N/A dup=2 drop=0
video:152kB audio:1699kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
[libx264 @ 0x7ff19d032200] frame I:12 Avg QP: 8.62 size: 5725
[libx264 @ 0x7ff19d032200] frame P:698 Avg QP: 9.15 size: 38
[libx264 @ 0x7ff19d032200] frame B:2059 Avg QP:12.67 size: 29
[libx264 @ 0x7ff19d032200] consecutive B-frames: 0.8% 0.1% 0.0% 99.1%
[libx264 @ 0x7ff19d032200] mb I I16..4: 81.6% 12.5% 6.0%
[libx264 @ 0x7ff19d032200] mb P I16..4: 0.0% 0.0% 0.0% P16..4: 0.0% 0.0% 0.0% 0.0% 0.0% skip:100.0%
[libx264 @ 0x7ff19d032200] mb B I16..4: 0.0% 0.0% 0.0% B16..8: 0.0% 0.0% 0.0% direct: 0.0% skip:100.0% L0:84.5% L1:15.5% BI: 0.0%
[libx264 @ 0x7ff19d032200] 8x8 transform intra:12.5% inter:21.1%
[libx264 @ 0x7ff19d032200] coded y,uvDC,uvAC intra: 3.7% 0.0% 0.0% inter: 0.0% 0.0% 0.0%
[libx264 @ 0x7ff19d032200] i16 v,h,dc,p: 97% 1% 3% 0%
[libx264 @ 0x7ff19d032200] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 57% 2% 40% 1% 0% 0% 0% 0% 0%
[libx264 @ 0x7ff19d032200] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 38% 13% 20% 8% 4% 5% 3% 5% 4%
[libx264 @ 0x7ff19d032200] i8c dc,h,v,p: 100% 0% 0% 0%
[libx264 @ 0x7ff19d032200] Weighted P-Frames: Y:0.0% UV:0.0%
[libx264 @ 0x7ff19d032200] ref P L0: 96.2% 1.3% 2.6%
[libx264 @ 0x7ff19d032200] ref B L0: 17.5% 82.5% 0.1%
[libx264 @ 0x7ff19d032200] ref B L1: 98.2% 1.8%
[libx264 @ 0x7ff19d032200] kb/s:11.25

## segment 11.ts is bad, here is some output

$ ffmpeg -v warning -i 000011.ts -f null -
[NULL @ 0x7fc814025e00] start time for stream 1 is not set in estimate_timings_from_pts
[mpegts @ 0x7fc814000000] Could not find codec parameters for stream 1 (Audio: aac ([15][0][0][0] / 0x000F), 0 channels, fltp): unspecified sample rate
Consider increasing the value for the 'analyzeduration' and 'probesize' options
[null @ 0x7fc81300a800] Encoder did not produce proper pts, making some up.

$ ffprobe 000011.ts
ffprobe version 2.8.4 Copyright (c) 2007-2015 the FFmpeg developers

built with Apple LLVM version 7.0.2 (clang-700.1.81)
configuration: --prefix=/usr/local/Cellar/ffmpeg/2.8.4 --enable-shared --enable-pthreads --enable-gpl --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-opencl --enable-libx264 --enable-libmp3lame --enable-libvo-aacenc --enable-libxvid --enable-vda
libavutil 54. 31.100 / 54. 31.100
libavcodec 56. 60.100 / 56. 60.100
libavformat 56. 40.101 / 56. 40.101
libavdevice 56. 4.100 / 56. 4.100
libavfilter 5. 40.101 / 5. 40.101
libavresample 2. 1. 0 / 2. 1. 0
libswscale 3. 1.101 / 3. 1.101
libswresample 1. 2.101 / 1. 2.101
libpostproc 53. 3.100 / 53. 3.100

[NULL @ 0x7fa09a028800] start time for stream 1 is not set in estimate_timings_from_pts
[mpegts @ 0x7fa09a002a00] Could not find codec parameters for stream 1 (Audio: aac ([15][0][0][0] / 0x000F), 0 channels, fltp): unspecified sample rate
Consider increasing the value for the 'analyzeduration' and 'probesize' options
Input #0, mpegts, from '000011.ts':

Duration: 00:00:00.76, start: 1.480000, bitrate: 102 kb/s
Program 1

Metadata:

service_name : Track 7
service_provider: FFmpeg

Stream #0:0[0x100]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p, 768x768, 25 fps, 25 tbr, 90k tbn, 50 tbc
Stream #0:1[0x101](und): Audio: aac ([15][0][0][0] / 0x000F), 0 channels, fltp

## Second experiment

I then modified my process to not include the watermark image. I found the resulting mp4 had a shorter duration than the output of first attempt. I then converted
it to mpeg ts and saw it was one segment less. All the segments had an audio stream and played just fine in vlc and quicktime.

ffmpeg -i mack_the_knife.mp3 -shortest out2.mp4 > third.log 2>&1
ffmpeg -i out2.mp4 -vcodec libx264 -bsf:v h264_mp4toannexb -strict strict -flags -global_header -dcodec copy -acodec copy -q:v 0 -map 0 -f segment -segment_list play2.m3u8 -segment_time 10 -segment_list_flags +live -reset_timestamps 1 -segment_list_entry_prefix '​http://localhost:8000/' ./%06d_2.ts > fourth.log 2>&1
ffprobe -v error -show_entries format=duration -of default=noprint_wrappers=1:nokey=1 out2.mp4
ffmpeg -v warning -i 000000_2.ts -f null - > verify_good_segment_2.log 2>&1
# of note there is no 11_2.ts
ffmpeg -v warning -i 000010_2.ts -f null - > verify_good_segment_3.log 2>&1

This suggests to me that my mp3 is fine. The image is just an image so I don't know why it would change things so much.

## Third experiment

The mp4 duration changing concerned me. I then altered the first situation to specify the exact duration of the mp3 as reported by ffprobe.
The resulting mp4 had a duration closer to the mp3. I then converted the mp4 to mpeg ts and saw it was one segment less than the first attempt. The file was playable.

ffmpeg -i mack_the_knife.mp3 -loop 1 -i watermark.jpg -pix_fmt yuv420p -t 108.127714 out3.mp4 > fifth.log 2>&1
ffmpeg -i out3.mp4 -vcodec libx264 -bsf:v h264_mp4toannexb -strict strict -flags -global_header -dcodec copy -acodec copy -q:v 0 -map 0 -f segment -segment_list play3.m3u8 -segment_time 10 -segment_list_flags +live -reset_timestamps 1 -segment_list_entry_prefix '​http://localhost:8000/' ./%06d_3.ts > sixth.log 2>&1
ffmpeg -v warning -i 000000_3.ts -f null - > verify_good_segment_4.log 2>&1
ffmpeg -v warning -i 000011_3.ts -f null - > verify_good_segment_5.log 2>&1

[Carl Eugen Hoyos]

Please always test current FFmpeg git head before reporting issues here.
Either use -vcodec copy or the bitstream filter, I don't think using both is supported.
What are you expecting to find in the last segment?