#3902 closed defect (duplicate)
FFmpeg MD5 output different with same data #3
Reported by: | ahthovaikied | Owned by: | |
---|---|---|---|
Priority: | normal | Component: | undetermined |
Version: | git-master | Keywords: | mpeg2video |
Cc: | Blocked By: | ||
Blocking: | Reproduced by developer: | no | |
Analyzed by developer: | no |
Description
Computing the MD5 of media streams produce different results between two of my machines, with same input data.
See https://trac.ffmpeg.org/ticket/3871 for some background.
The 2 configurations are :
- Ubuntu 14.04, Core i7 950, FFmpeg compiled with:
--enable-gpl --enable-version3 --enable-nonfree --disable-runtime-cpudetect --disable-ffserver --disable-ffplay --disable-encoders --disable-decoders --disable-filters --disable-debug --cpu=corei7
- Ubuntu 14.04, Atom D525, FFmpeg compiled with same as previous but
--cpu=atom
instead of--cpu=corei7
I can only reproduce this with some Matroska files which are direct DVD rip (MPEG2 + AC3, no transcoding) made with MakeMKV (http://www.makemkv.com/), but not all.
I get the same behaviour with FFmpeg 2.3.3 and from git master.
Taking video and audio streams:
- corei7
$ ffmpeg -loglevel quiet -i test.mkv -map v -map a -c:v copy -c:a copy -f md5 - MD5=1337b0c18400b795764abf7569e37d35
- atom
$ ffmpeg -loglevel quiet -i test.mkv -map v -map a -c:v copy -c:a copy -f md5 - MD5=23e468f0cf9671ffcd1f6478ae052220
...MD5 do not match.
Taking only video stream:
- corei7
$ ffmpeg -loglevel quiet -i test.mkv -map v -c:v copy -f md5 - MD5=d61c1256650a32b9acf72b57810b0ded
- atom
$ ffmpeg -loglevel quiet -i test.mkv -map v -c:v copy -f md5 - MD5=d61c1256650a32b9acf72b57810b0ded
...MD5 do match.
Taking only audio streams:
- corei7
$ ffmpeg -loglevel quiet -i test.mkv -map a -c:a copy -f md5 - MD5=112d7f2c095285d682b189d68cd13e1a
- atom
$ ffmpeg -loglevel quiet -i test.mkv -map a -c:a copy -f md5 - MD5=112d7f2c095285d682b189d68cd13e1a
...MD5 do match.
Full sample output:
$ ffmpeg -i test.mkv -map v -map a -c:v copy -c:a copy -f md5 - ffmpeg version N-65928-g7997d78 Copyright (c) 2000-2014 the FFmpeg developers built on Aug 27 2014 20:45:49 with gcc 4.8 (Ubuntu 4.8.2-19ubuntu1) configuration: --enable-gpl --enable-version3 --enable-nonfree --disable-runtime-cpudetect --disable-ffserver --disable-ffplay --disable-encoders --disable-decoders --disable-filters --disable-debug --cpu=corei7 libavutil 54. 7.100 / 54. 7.100 libavcodec 56. 0.101 / 56. 0.101 libavformat 56. 3.100 / 56. 3.100 libavdevice 56. 0.100 / 56. 0.100 libavfilter 5. 0.103 / 5. 0.103 libswscale 3. 0.100 / 3. 0.100 libswresample 1. 1.100 / 1. 1.100 libpostproc 53. 0.100 / 53. 0.100 [matroska,webm @ 0x1981ca0] Could not find codec parameters for stream 16 (Attachment: none): unknown codec Consider increasing the value for the 'analyzeduration' and 'probesize' options Input #0, matroska,webm, from 'test.mkv': Metadata: title : Goodfellas encoder : libebml v1.2.2 + libmatroska v1.3.0 creation_time : 2013-01-17 21:50:53 Duration: 02:19:28.66, start: 0.024000, bitrate: 6479 kb/s Chapter #0.0: start 0.000000, end 155.280000 Metadata: title : Chapter 01 Chapter #0.1: start 155.280000, end 334.320000 Metadata: title : Chapter 02 Chapter #0.2: start 334.320000, end 437.040000 Metadata: title : Chapter 03 Chapter #0.3: start 437.040000, end 525.160000 Metadata: title : Chapter 04 Chapter #0.4: start 525.160000, end 621.560000 Metadata: title : Chapter 05 Chapter #0.5: start 621.560000, end 774.600000 Metadata: title : Chapter 06 Chapter #0.6: start 774.600000, end 864.120000 Metadata: title : Chapter 07 Chapter #0.7: start 864.120000, end 956.800000 Metadata: title : Chapter 08 Chapter #0.8: start 956.800000, end 1133.880000 Metadata: title : Chapter 09 Chapter #0.9: start 1133.880000, end 1367.160000 Metadata: title : Chapter 10 Chapter #0.10: start 1367.160000, end 1648.480000 Metadata: title : Chapter 11 Chapter #0.11: start 1648.480000, end 1814.360000 Metadata: title : Chapter 12 Chapter #0.12: start 1814.360000, end 1990.440000 Metadata: title : Chapter 13 Chapter #0.13: start 1990.440000, end 2054.880000 Metadata: title : Chapter 14 Chapter #0.14: start 2054.880000, end 2148.680000 Metadata: title : Chapter 15 Chapter #0.15: start 2148.680000, end 2384.920000 Metadata: title : Chapter 16 Chapter #0.16: start 2384.920000, end 2556.320000 Metadata: title : Chapter 17 Chapter #0.17: start 2556.320000, end 2815.000000 Metadata: title : Chapter 18 Chapter #0.18: start 2815.000000, end 2995.680000 Metadata: title : Chapter 19 Chapter #0.19: start 2995.680000, end 3286.240000 Metadata: title : Chapter 20 Chapter #0.20: start 3286.240000, end 3519.080000 Metadata: title : Chapter 21 Chapter #0.21: start 3519.080000, end 3668.720000 Metadata: title : Chapter 22 Chapter #0.22: start 3668.720000, end 3819.240000 Metadata: title : Chapter 23 Chapter #0.23: start 3819.240000, end 3896.200000 Metadata: title : Chapter 24 Chapter #0.24: start 3896.200000, end 4173.280000 Metadata: title : Chapter 25 Chapter #0.25: start 4173.280000, end 4356.320000 Metadata: title : Chapter 26 Chapter #0.26: start 4356.320000, end 4525.160000 Metadata: title : Chapter 27 Chapter #0.27: start 4525.160000, end 4605.320000 Metadata: title : Chapter 28 Chapter #0.28: start 4605.320000, end 4780.360000 Metadata: title : Chapter 29 Chapter #0.29: start 4780.360000, end 4926.600000 Metadata: title : Chapter 30 Chapter #0.30: start 4926.600000, end 5076.760000 Metadata: title : Chapter 31 Chapter #0.31: start 5076.760000, end 5206.480000 Metadata: title : Chapter 32 Chapter #0.32: start 5206.480000, end 5370.640000 Metadata: title : Chapter 33 Chapter #0.33: start 5370.640000, end 5631.080000 Metadata: title : Chapter 34 Chapter #0.34: start 5631.080000, end 5755.200000 Metadata: title : Chapter 35 Chapter #0.35: start 5755.200000, end 5917.920000 Metadata: title : Chapter 36 Chapter #0.36: start 5917.920000, end 6131.360000 Metadata: title : Chapter 37 Chapter #0.37: start 6131.360000, end 6284.880000 Metadata: title : Chapter 38 Chapter #0.38: start 6284.880000, end 6522.320000 Metadata: title : Chapter 39 Chapter #0.39: start 6522.320000, end 7130.760000 Metadata: title : Chapter 40 Chapter #0.40: start 7130.760000, end 7325.080000 Metadata: title : Chapter 41 Chapter #0.41: start 7325.080000, end 7450.720000 Metadata: title : Chapter 42 Chapter #0.42: start 7450.720000, end 7595.760000 Metadata: title : Chapter 43 Chapter #0.43: start 7595.760000, end 7769.480000 Metadata: title : Chapter 44 Chapter #0.44: start 7769.480000, end 7985.560000 Metadata: title : Chapter 45 Chapter #0.45: start 7985.560000, end 8086.640000 Metadata: title : Chapter 46 Chapter #0.46: start 8086.640000, end 8368.680000 Metadata: title : Chapter 47 Stream #0:0(eng): Video: mpeg2video (Main), yuv420p(tv), 720x576 [SAR 64:45 DAR 16:9], max. 7500 kb/s, 25 fps, 25 tbr, 1k tbn, 50 tbc (default) Stream #0:1(eng): Audio: ac3, 48000 Hz, 5.1(side), 384 kb/s (default) Stream #0:2(fre): Audio: ac3, 48000 Hz, stereo, 192 kb/s Stream #0:3(ita): Audio: ac3, 48000 Hz, stereo, 192 kb/s Stream #0:4(eng): Audio: ac3, 48000 Hz, stereo, 192 kb/s Metadata: title : 2/0 Stream #0:5(eng): Subtitle: dvd_subtitle Stream #0:6(fre): Subtitle: dvd_subtitle Stream #0:7(ita): Subtitle: dvd_subtitle Stream #0:8(ger): Subtitle: dvd_subtitle Stream #0:9(spa): Subtitle: dvd_subtitle Stream #0:10(dut): Subtitle: dvd_subtitle Stream #0:11(ara): Subtitle: dvd_subtitle Stream #0:12(bul): Subtitle: dvd_subtitle Stream #0:13(rum): Subtitle: dvd_subtitle Stream #0:14(eng): Subtitle: dvd_subtitle Stream #0:15(ita): Subtitle: dvd_subtitle Stream #0:16: Attachment: none Metadata: filename : media_md5.bin mimetype : application/octet-stream Output #0, md5, to 'pipe:': Metadata: title : Goodfellas encoder : Lavf56.3.100 Chapter #0.0: start 0.000000, end 155.256000 Metadata: title : Chapter 01 Chapter #0.1: start 155.256000, end 334.296000 Metadata: title : Chapter 02 Chapter #0.2: start 334.296000, end 437.016000 Metadata: title : Chapter 03 Chapter #0.3: start 437.016000, end 525.136000 Metadata: title : Chapter 04 Chapter #0.4: start 525.136000, end 621.536000 Metadata: title : Chapter 05 Chapter #0.5: start 621.536000, end 774.576000 Metadata: title : Chapter 06 Chapter #0.6: start 774.576000, end 864.096000 Metadata: title : Chapter 07 Chapter #0.7: start 864.096000, end 956.776000 Metadata: title : Chapter 08 Chapter #0.8: start 956.776000, end 1133.856000 Metadata: title : Chapter 09 Chapter #0.9: start 1133.856000, end 1367.136000 Metadata: title : Chapter 10 Chapter #0.10: start 1367.136000, end 1648.456000 Metadata: title : Chapter 11 Chapter #0.11: start 1648.456000, end 1814.336000 Metadata: title : Chapter 12 Chapter #0.12: start 1814.336000, end 1990.416000 Metadata: title : Chapter 13 Chapter #0.13: start 1990.416000, end 2054.856000 Metadata: title : Chapter 14 Chapter #0.14: start 2054.856000, end 2148.656000 Metadata: title : Chapter 15 Chapter #0.15: start 2148.656000, end 2384.896000 Metadata: title : Chapter 16 Chapter #0.16: start 2384.896000, end 2556.296000 Metadata: title : Chapter 17 Chapter #0.17: start 2556.296000, end 2814.976000 Metadata: title : Chapter 18 Chapter #0.18: start 2814.976000, end 2995.656000 Metadata: title : Chapter 19 Chapter #0.19: start 2995.656000, end 3286.216000 Metadata: title : Chapter 20 Chapter #0.20: start 3286.216000, end 3519.056000 Metadata: title : Chapter 21 Chapter #0.21: start 3519.056000, end 3668.696000 Metadata: title : Chapter 22 Chapter #0.22: start 3668.696000, end 3819.216000 Metadata: title : Chapter 23 Chapter #0.23: start 3819.216000, end 3896.176000 Metadata: title : Chapter 24 Chapter #0.24: start 3896.176000, end 4173.256000 Metadata: title : Chapter 25 Chapter #0.25: start 4173.256000, end 4356.296000 Metadata: title : Chapter 26 Chapter #0.26: start 4356.296000, end 4525.136000 Metadata: title : Chapter 27 Chapter #0.27: start 4525.136000, end 4605.296000 Metadata: title : Chapter 28 Chapter #0.28: start 4605.296000, end 4780.336000 Metadata: title : Chapter 29 Chapter #0.29: start 4780.336000, end 4926.576000 Metadata: title : Chapter 30 Chapter #0.30: start 4926.576000, end 5076.736000 Metadata: title : Chapter 31 Chapter #0.31: start 5076.736000, end 5206.456000 Metadata: title : Chapter 32 Chapter #0.32: start 5206.456000, end 5370.616000 Metadata: title : Chapter 33 Chapter #0.33: start 5370.616000, end 5631.056000 Metadata: title : Chapter 34 Chapter #0.34: start 5631.056000, end 5755.176000 Metadata: title : Chapter 35 Chapter #0.35: start 5755.176000, end 5917.896000 Metadata: title : Chapter 36 Chapter #0.36: start 5917.896000, end 6131.336000 Metadata: title : Chapter 37 Chapter #0.37: start 6131.336000, end 6284.856000 Metadata: title : Chapter 38 Chapter #0.38: start 6284.856000, end 6522.296000 Metadata: title : Chapter 39 Chapter #0.39: start 6522.296000, end 7130.736000 Metadata: title : Chapter 40 Chapter #0.40: start 7130.736000, end 7325.056000 Metadata: title : Chapter 41 Chapter #0.41: start 7325.056000, end 7450.696000 Metadata: title : Chapter 42 Chapter #0.42: start 7450.696000, end 7595.736000 Metadata: title : Chapter 43 Chapter #0.43: start 7595.736000, end 7769.456000 Metadata: title : Chapter 44 Chapter #0.44: start 7769.456000, end 7985.536000 Metadata: title : Chapter 45 Chapter #0.45: start 7985.536000, end 8086.616000 Metadata: title : Chapter 46 Chapter #0.46: start 8086.616000, end 8368.656000 Metadata: title : Chapter 47 Stream #0:0(eng): Video: mpeg2video, yuv420p, 720x576 [SAR 64:45 DAR 16:9], q=2-31, max. 7500 kb/s, 25 fps, 25 tbn, 25 tbc (default) Stream #0:1(eng): Audio: ac3, 48000 Hz, 5.1(side), 384 kb/s (default) Stream #0:2(fre): Audio: ac3, 48000 Hz, stereo, 192 kb/s Stream #0:3(ita): Audio: ac3, 48000 Hz, stereo, 192 kb/s Stream #0:4(eng): Audio: ac3, 48000 Hz, stereo, 192 kb/s Metadata: title : 2/0 Stream mapping: Stream #0:0 -> #0:0 (copy) Stream #0:1 -> #0:1 (copy) Stream #0:2 -> #0:2 (copy) Stream #0:3 -> #0:3 (copy) Stream #0:4 -> #0:4 (copy) Press [q] to stop, [?] for help MD5=1337b0c18400b795764abf7569e37d35 0kB time=02:16:22.62 bitrate= 0.0kbits/s frame=209217 fps=10775 q=-1.0 Lsize= 0kB time=02:19:28.64 bitrate= 0.0kbits/s video:5607170kB audio:980700kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
I can not upload the file because as you can expect for a DVD rip it is quite big.
Change History (44)
comment:1 by , 10 years ago
comment:3 by , 10 years ago
Replying to cehoyos:
Did you test with identical configure options?
I just did the test (on the cut file), same behavior :
corei7:
$ ffmpeg -i cut.mkv -map v -map a -c:v copy -c:a copy -f md5 - ffmpeg version N-65928-g7997d78 Copyright (c) 2000-2014 the FFmpeg developers built on Aug 27 2014 22:26:16 with gcc 4.8 (Ubuntu 4.8.2-19ubuntu1) configuration: --enable-gpl --enable-version3 --enable-nonfree --disable-runtime-cpudetect --disable-ffserver --disable-ffplay --disable-encoders --disable-decoders --disable-filters --disable-debug libavutil 54. 7.100 / 54. 7.100 libavcodec 56. 0.101 / 56. 0.101 libavformat 56. 3.100 / 56. 3.100 libavdevice 56. 0.100 / 56. 0.100 libavfilter 5. 0.103 / 5. 0.103 libswscale 3. 0.100 / 3. 0.100 libswresample 1. 1.100 / 1. 1.100 libpostproc 53. 0.100 / 53. 0.100 [matroska,webm @ 0x29ccc80] Could not find codec parameters for stream 16 (Attachment: none): unknown codec Consider increasing the value for the 'analyzeduration' and 'probesize' options Input #0, matroska,webm, from 'cut.mkv': Metadata: title : Goodfellas ENCODER : Lavf55.48.100 Duration: 00:00:10.65, start: 0.000000, bitrate: 5539 kb/s Chapter #0.0: start 0.000000, end 10.000000 Metadata: title : Chapter 06 Stream #0:0(eng): Video: mpeg2video (Main), yuv420p(tv), 720x576 [SAR 64:45 DAR 16:9], max. 7500 kb/s, 25 fps, 25 tbr, 1k tbn, 50 tbc (default) Stream #0:1(eng): Audio: ac3, 48000 Hz, 5.1(side), 384 kb/s (default) Stream #0:2(fre): Audio: ac3, 48000 Hz, stereo, 192 kb/s Stream #0:3(ita): Audio: ac3, 48000 Hz, stereo, 192 kb/s Stream #0:4(eng): Audio: ac3, 48000 Hz, stereo, 192 kb/s Metadata: title : 2/0 LANGUAGE : eng Stream #0:5(eng): Subtitle: dvd_subtitle Stream #0:6(fre): Subtitle: dvd_subtitle Stream #0:7(ita): Subtitle: dvd_subtitle Stream #0:8(ger): Subtitle: dvd_subtitle Stream #0:9(spa): Subtitle: dvd_subtitle Stream #0:10(dut): Subtitle: dvd_subtitle Stream #0:11(ara): Subtitle: dvd_subtitle Stream #0:12(bul): Subtitle: dvd_subtitle Stream #0:13(rum): Subtitle: dvd_subtitle Stream #0:14(eng): Subtitle: dvd_subtitle Stream #0:15(ita): Subtitle: dvd_subtitle Stream #0:16: Attachment: none Metadata: filename : media_md5.bin mimetype : application/octet-stream Output #0, md5, to 'pipe:': Metadata: title : Goodfellas encoder : Lavf56.3.100 Chapter #0.0: start 0.000000, end 10.000000 Metadata: title : Chapter 06 Stream #0:0(eng): Video: mpeg2video, yuv420p, 720x576 [SAR 64:45 DAR 16:9], q=2-31, max. 7500 kb/s, 25 fps, 25 tbn, 25 tbc (default) Stream #0:1(eng): Audio: ac3, 48000 Hz, 5.1(side), 384 kb/s (default) Stream #0:2(fre): Audio: ac3, 48000 Hz, stereo, 192 kb/s Stream #0:3(ita): Audio: ac3, 48000 Hz, stereo, 192 kb/s Stream #0:4(eng): Audio: ac3, 48000 Hz, stereo, 192 kb/s Metadata: title : 2/0 LANGUAGE : eng Stream mapping: Stream #0:0 -> #0:0 (copy) Stream #0:1 -> #0:1 (copy) Stream #0:2 -> #0:2 (copy) Stream #0:3 -> #0:3 (copy) Stream #0:4 -> #0:4 (copy) Press [q] to stop, [?] for help MD5=0b0257acc6662c99b71e879d27705ce9 frame= 240 fps=0.0 q=-1.0 Lsize= 0kB time=00:00:10.01 bitrate= 0.0kbits/s video:5912kB audio:1174kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
atom:
$ ffmpeg -i cut.mkv -map v -map a -c:v copy -c:a copy -f md5 - ffmpeg version N-65930-ga59d922 Copyright (c) 2000-2014 the FFmpeg developers built on Aug 27 2014 22:35:18 with gcc 4.8 (Ubuntu 4.8.2-19ubuntu1) configuration: --enable-gpl --enable-version3 --enable-nonfree --disable-runtime-cpudetect --disable-ffserver --disable-ffplay --disable-encoders --disable-decoders --disable-filters --disable-debug libavutil 54. 7.100 / 54. 7.100 libavcodec 56. 0.101 / 56. 0.101 libavformat 56. 3.100 / 56. 3.100 libavdevice 56. 0.100 / 56. 0.100 libavfilter 5. 0.103 / 5. 0.103 libswscale 3. 0.100 / 3. 0.100 libswresample 1. 1.100 / 1. 1.100 libpostproc 53. 0.100 / 53. 0.100 [matroska,webm @ 0x22f8c80] Could not find codec parameters for stream 16 (Attachment: none): unknown codec Consider increasing the value for the 'analyzeduration' and 'probesize' options Input #0, matroska,webm, from 'cut.mkv': Metadata: title : Goodfellas ENCODER : Lavf55.48.100 Duration: 00:00:10.65, start: 0.000000, bitrate: 5539 kb/s Chapter #0.0: start 0.000000, end 10.000000 Metadata: title : Chapter 06 Stream #0:0(eng): Video: mpeg2video, 720x576, max. 7500 kb/s, SAR 64:45 DAR 16:9, 25 fps, 25 tbr, 1k tbn, 50 tbc (default) Stream #0:1(eng): Audio: ac3, 48000 Hz, 5.1(side), 384 kb/s (default) Stream #0:2(fre): Audio: ac3, 48000 Hz, stereo, 192 kb/s Stream #0:3(ita): Audio: ac3, 48000 Hz, stereo, 192 kb/s Stream #0:4(eng): Audio: ac3, 48000 Hz, stereo, 192 kb/s Metadata: title : 2/0 LANGUAGE : eng Stream #0:5(eng): Subtitle: dvd_subtitle Stream #0:6(fre): Subtitle: dvd_subtitle Stream #0:7(ita): Subtitle: dvd_subtitle Stream #0:8(ger): Subtitle: dvd_subtitle Stream #0:9(spa): Subtitle: dvd_subtitle Stream #0:10(dut): Subtitle: dvd_subtitle Stream #0:11(ara): Subtitle: dvd_subtitle Stream #0:12(bul): Subtitle: dvd_subtitle Stream #0:13(rum): Subtitle: dvd_subtitle Stream #0:14(eng): Subtitle: dvd_subtitle Stream #0:15(ita): Subtitle: dvd_subtitle Stream #0:16: Attachment: none Metadata: filename : media_md5.bin mimetype : application/octet-stream Output #0, md5, to 'pipe:': Metadata: title : Goodfellas encoder : Lavf56.3.100 Chapter #0.0: start 0.000000, end 10.000000 Metadata: title : Chapter 06 Stream #0:0(eng): Video: mpeg2video, 720x576 [SAR 64:45 DAR 16:9], q=2-31, max. 7500 kb/s, 25 fps, 50 tbn, 50 tbc (default) Stream #0:1(eng): Audio: ac3, 48000 Hz, 5.1(side), 384 kb/s (default) Stream #0:2(fre): Audio: ac3, 48000 Hz, stereo, 192 kb/s Stream #0:3(ita): Audio: ac3, 48000 Hz, stereo, 192 kb/s Stream #0:4(eng): Audio: ac3, 48000 Hz, stereo, 192 kb/s Metadata: title : 2/0 LANGUAGE : eng Stream mapping: Stream #0:0 -> #0:0 (copy) Stream #0:1 -> #0:1 (copy) Stream #0:2 -> #0:2 (copy) Stream #0:3 -> #0:3 (copy) Stream #0:4 -> #0:4 (copy) Press [q] to stop, [?] for help MD5=726d4f2a3d79e000e0bbde6397311bb2 frame= 240 fps=0.0 q=-1.0 Lsize= 0kB time=00:00:10.01 bitrate= 0.0kbits/s video:5912kB audio:1174kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
comment:5 by , 10 years ago
follow-up: 7 comment:6 by , 10 years ago
Please test running the same ffmpeg
binary on both systems.
follow-up: 33 comment:7 by , 10 years ago
Replying to cehoyos:
Please test running the same
ffmpeg
binary on both systems.
I had some difficulty to build a binary portable across the 2 machines, and had to change the build configuration by adding --disable-hwaccels --disable-devices --disable-vdpau
.
Full configure line is now:
--enable-gpl --enable-version3 --enable-nonfree --disable-runtime-cpudetect --disable-ffserver --disable-ffplay --disable-encoders --disable-decoders --disable-filters --disable-debug --disable-hwaccels --disable-devices --disable-vdpau
By the way, isn't this a bug that I require both --disable-hwaccels
and --disable-vdpau
to not link against libvdpau?
I built the binary on the corei7 machine, then rsynced the binary to the atom, and...
$ ffmpeg -loglevel quiet -i cut.mkv -map v -map a -c copy -f md5 - MD5=726d4f2a3d79e000e0bbde6397311bb2
...MD5 is now the same on both machines.
What is you interpretation of this?
comment:8 by , 10 years ago
I did some more testing and it seems the MD5 start to differ when FFmpeg is linked with both libvaapi and libvdpau.
On the corei7:
- with base configuration +
--disable-vaapi --disable-vdpau
, I get 726d4f2a3d79e000e0bbde6397311bb2 like the atom => looks like the right "reference" MD5 - with base configuration +
--disable-vaapi
, I get 0b0257acc6662c99b71e879d27705ce9 - with base configuration +
--disable-vdpau
, I get 0b0257acc6662c99b71e879d27705ce9
What do you think?
Also I don't understand how these libraries are involved in the demuxing process.
follow-up: 11 comment:10 by , 10 years ago
Replying to cehoyos:
Did you already test
--disable-pthreads
?
No, but -thread 1
did not change anything.
I found out that older FFmpeg versions produce the same MD5 when linking against libvaapi and libvdpau and when not.
I ran a git bisect like you taught me ;) and the regression comes from this commit:
commit afa93d198aaf2cc661c4df6d4095cd030265d30a Author: Yusuke Nakamura <email_removed> Date: Sun Oct 27 11:07:43 2013 +0100 hevc_parser: Set pict_type, key_frame and output_picture_number. Conflicts: libavcodec/hevc.c
follow-up: 12 comment:11 by , 10 years ago
Replying to ahthovaikied:
and the regression comes from this commit:
commit afa93d198aaf2cc661c4df6d4095cd030265d30a
So disabling the hevc parser fixes the issue?
comment:12 by , 10 years ago
Replying to cehoyos:
Replying to ahthovaikied:
and the regression comes from this commit:
commit afa93d198aaf2cc661c4df6d4095cd030265d30aSo disabling the hevc parser fixes the issue?
I tried --disable-parser=hevc
, but it does not fix it...
comment:14 by , 10 years ago
comment:16 by , 10 years ago
Replying to cehoyos:
Is the issue reproducible with
-an
or-vn
?
Please read my first post, I already tried isolating video or audio streams.
follow-up: 20 comment:17 by , 10 years ago
Then please repeat your findings in one sentence, this is a long ticket (and it isn't clear to anybody what you are trying to prove).
comment:18 by , 10 years ago
Next step will be to test with -f framecrc
but I would like to narrow the issue down first.
follow-up: 23 comment:19 by , 10 years ago
I would suggest to test with ffprobe -show_data
(or -show_data_hash
): that would allow to find out whether the difference is in demuxing or in the treatments that the command-line tool ffmpeg
applies to the content.
comment:20 by , 10 years ago
Replying to cehoyos:
Then please repeat your findings in one sentence, this is a long ticket (and it isn't clear to anybody what you are trying to prove).
I understand you have limited available time, and appreciate you taking some to help me, but you have asked the same question several times without even reading my answer...
This is not a simple problem, I can not make it one.
Anyway here is the shortest summary I can make:
- I have some files (MPEG2 video + AC3 audio), for which the command:
ffmpeg -i input.mkv -map v -map a -c copy -f md5 -
returns different MD5 between 2 FFmpeg builds of the same version, with the same configure options on 2 different machines - A short file to reproduce the problem has been uploaded on FFmpeg's FTP
- The problem can only be seen when selecting video and audio streams, with only video or only audio, the MD5 matches
-thread 1
or build with--disable-pthreads
does not change anything- After trying to use the same exact binary on both machines, I found out the difference is introduced by the fact thah one of the build is linking against vdpau and vaapi
- When building with
--disable-vaapi --disable-vdpau
, the difference of MD5 is gone - When building with only
--disable-vaapi
or--disable-vdpau
, there is a difference of MD5 - I tried going back to older FFmpeg release, and they produce the same MD5 with
--disable-vaapi --disable-vdpau
or without - I ran a git bisect, and the weird behavior started at commit afa93d198aaf2cc661c4df6d4095cd030265d30a
- Building with
--disable-parser=hevc
does not eliminate the difference of MD5 for recent FFmpeg versions
comment:22 by , 10 years ago
Did you already test with valgrind to find out if using uninitialised memory is the culprit?
comment:23 by , 10 years ago
Replying to Cigaes:
I would suggest to test with
ffprobe -show_data
(or-show_data_hash
): that would allow to find out whether the difference is in demuxing or in the treatments that the command-line toolffmpeg
applies to the content.
Without --disable-vaapi --disable-vdpau
:
$ ffprobe -show_streams -show_data_hash md5 cut.mkv 2>&1 | grep -F 'extradata_hash=MD5:' extradata_hash=MD5:2ec9750f9d09a451409c18ae647aacc6 extradata_hash=MD5:d41d8cd98f00b204e9800998ecf8427e extradata_hash=MD5:d41d8cd98f00b204e9800998ecf8427e extradata_hash=MD5:d41d8cd98f00b204e9800998ecf8427e extradata_hash=MD5:d41d8cd98f00b204e9800998ecf8427e extradata_hash=MD5:99cfb7dc1fc487952a318f80affcf026 extradata_hash=MD5:99cfb7dc1fc487952a318f80affcf026 extradata_hash=MD5:99cfb7dc1fc487952a318f80affcf026 extradata_hash=MD5:99cfb7dc1fc487952a318f80affcf026 extradata_hash=MD5:99cfb7dc1fc487952a318f80affcf026 extradata_hash=MD5:99cfb7dc1fc487952a318f80affcf026 extradata_hash=MD5:99cfb7dc1fc487952a318f80affcf026 extradata_hash=MD5:99cfb7dc1fc487952a318f80affcf026 extradata_hash=MD5:99cfb7dc1fc487952a318f80affcf026 extradata_hash=MD5:99cfb7dc1fc487952a318f80affcf026 extradata_hash=MD5:99cfb7dc1fc487952a318f80affcf026 extradata_hash=MD5:0707dbbdde6509fc31a7afab98ac3ebd
With --disable-vaapi --disable-vdpau
:
$ ffprobe -show_streams -show_data_hash md5 cut.mkv 2>&1 | grep -F 'extradata_hash=MD5:' extradata_hash=MD5:2ec9750f9d09a451409c18ae647aacc6 extradata_hash=MD5:d41d8cd98f00b204e9800998ecf8427e extradata_hash=MD5:d41d8cd98f00b204e9800998ecf8427e extradata_hash=MD5:d41d8cd98f00b204e9800998ecf8427e extradata_hash=MD5:d41d8cd98f00b204e9800998ecf8427e extradata_hash=MD5:99cfb7dc1fc487952a318f80affcf026 extradata_hash=MD5:99cfb7dc1fc487952a318f80affcf026 extradata_hash=MD5:99cfb7dc1fc487952a318f80affcf026 extradata_hash=MD5:99cfb7dc1fc487952a318f80affcf026 extradata_hash=MD5:99cfb7dc1fc487952a318f80affcf026 extradata_hash=MD5:99cfb7dc1fc487952a318f80affcf026 extradata_hash=MD5:99cfb7dc1fc487952a318f80affcf026 extradata_hash=MD5:99cfb7dc1fc487952a318f80affcf026 extradata_hash=MD5:99cfb7dc1fc487952a318f80affcf026 extradata_hash=MD5:99cfb7dc1fc487952a318f80affcf026 extradata_hash=MD5:99cfb7dc1fc487952a318f80affcf026 extradata_hash=MD5:0707dbbdde6509fc31a7afab98ac3ebd
I do not see the md5 in the output with -show_data_hash
, why is that?
EDIT: Fixed by using ffprobe -show_streams -show_data_hash md5 cut.mkv
.
EDIT2: Filter output for clarity.
follow-up: 25 comment:24 by , 10 years ago
Well, of course: you forgot to tell ffprobe
what data you want to show or hash: -show_packets
(for the packets' payload), -show_streams
(for the codecs' extradata).
comment:25 by , 10 years ago
Replying to Cigaes:
Well, of course: you forgot to tell
ffprobe
what data you want to show or hash:-show_packets
(for the packets' payload),-show_streams
(for the codecs' extradata).
Sorry, I have edited my previous post with the output of ffprobe -show_streams -show_data_hash md5 cut.mkv
comment:26 by , 10 years ago
Replying to cehoyos:
What if you only map one video and one audio stream?
MD5 are still different with -map v -map a:0
.
follow-up: 28 comment:27 by , 10 years ago
Good. Now you must try and spot differences in the extradata_hash=MD5:...
values. If not, you will need to do the same with -show_packets
. If you do not find differences, that means that the output part is producing different output, not the input part.
comment:28 by , 10 years ago
Replying to Cigaes:
Good. Now you must try and spot differences in the
extradata_hash=MD5:...
values. If not, you will need to do the same with-show_packets
. If you do not find differences, that means that the output part is producing different output, not the input part.
Without --disable-vaapi --disable-vdpau
:
$ ffprobe -show_packets -show_data_hash md5 cut.mkv 2>&1 | grep -F 'data_hash=MD5:' > /tmp/data_hashes1.txt
With --disable-vaapi --disable-vdpau
:
$ ffprobe -show_packets -show_data_hash md5 cut.mkv 2>&1 | grep -F 'data_hash=MD5:' > /tmp/data_hashes2.txt
No difference:
$ diff /tmp/data_hashes{1,2}.txt $
follow-up: 30 comment:29 by , 10 years ago
I don't know how nor why, but your bisecting looks suspicious to me. That commit only modified hevc code, and it seems unlikely to have any impact on audio or video or mpeg parsing.
There were H.264 parser modifications merged prior to that, try to check them maybe? No idea if they are bugfixes.
comment:30 by , 10 years ago
Replying to kurosu:
I don't know how nor why, but your bisecting looks suspicious to me. That commit only modified hevc code, and it seems unlikely to have any impact on audio or video or mpeg parsing.
You are absolutely right.
I checked again, and the git bisect run
stopped on this commit because the build failed with this tree, so my test script returned 1 (instead of 125 to tell git that this is untestable), and the commit was marked as the first "bad".
My apologies for the misleading information, I will do a git bisect again and post with my findings.
comment:31 by , 10 years ago
I don't know why, but when doing a bisect a very large proportion of versions fails to build with --disable-vaapi --disable-vdpau
(I's say 90%), with link errors like for example:
libavcodec/libavcodec.a(vc1dsp.o): dans la fonction « ff_vc1dsp_init »: vc1dsp.c:(.text.unlikely+0x38d): référence indéfinie vers « ff_vc1dsp_init_x86 » collect2: error: ld returned 1 exit status make: *** [ffmpeg_g] Erreur 1 make: *** Attente des tâches non terminées.... libavcodec/libavcodec.a(vc1dsp.o): dans la fonction « ff_vc1dsp_init »: vc1dsp.c:(.text.unlikely+0x38d): référence indéfinie vers « ff_vc1dsp_init_x86 » collect2: error: ld returned 1 exit status make: *** [ffplay_g] Erreur 1 libavcodec/libavcodec.a(vc1dsp.o): dans la fonction « ff_vc1dsp_init »: vc1dsp.c:(.text.unlikely+0x38d): référence indéfinie vers « ff_vc1dsp_init_x86 » collect2: error: ld returned 1 exit status make: *** [ffprobe_g] Erreur 1
I tried playing with configure options, but there is always a large number of build which fail and thus the bisect takes forever.
follow-ups: 34 36 comment:32 by , 10 years ago
I believe you are searching for 0dd4eff5
Please test 0dd4eff5 and 3788b8db to confirm.
comment:33 by , 10 years ago
Replying to ahthovaikied:
--enable-gpl --enable-version3 --enable-nonfree --disable-runtime-cpudetect --disable-ffserver --disable-ffplay --disable-encoders --disable-decoders --disable-filters --disable-debug --disable-hwaccels --disable-devices --disable-vdpau
Unrelated: Why are you using --enable-version3 --enable-nonfree
? What is this supposed to do? And please remove --enable-gpl
, it is only useful for decoders and filters but you disable both.
follow-up: 37 comment:34 by , 10 years ago
Replying to cehoyos:
I believe you are searching for 0dd4eff5
Probably not.
Which old version did not show a difference with the given configure line?
follow-up: 38 comment:35 by , 10 years ago
instead of -f md5 try to store the output in a file and find out with a hex editor what changes
comment:36 by , 10 years ago
Replying to cehoyos:
I believe you are searching for 0dd4eff5
Please test 0dd4eff5 and 3788b8db to confirm.
Both revisions are good (no difference of MD5).
comment:37 by , 10 years ago
Replying to cehoyos:
Which old version did not show a difference with the given configure line?
I know n2.1.5 is good.
comment:38 by , 10 years ago
Replying to michael:
instead of -f md5 try to store the output in a file and find out with a hex editor what changes
Without --disable-vaapi --disable-vdpau:
$ ffmpeg -loglevel quiet -i cut.mkv -map v -map a -c copy -f matroska /tmp/output1.mkv
With --disable-vaapi --disable-vdpau:
$ ffmpeg -loglevel quiet -i cut.mkv -map v -map a -c copy -f matroska /tmp/output2.mkv
Difference:
$ diff <(xxd -c 1 /tmp/output1.mkv) <(xxd -c 1 /tmp/output2.mkv) 350,365c350,365 < 000015d: 5d ] < 000015e: 9c . < 000015f: 9b . < 0000160: b1 . < 0000161: 6b k < 0000162: c4 . < 0000163: d5 . < 0000164: 23 # < 0000165: ce . < 0000166: 29 ) < 0000167: b1 . < 0000168: 3d = < 0000169: 66 f < 000016a: ec . < 000016b: ed . < 000016c: b2 . --- > 000015d: bd . > 000015e: b5 . > 000015f: e9 . > 0000160: 3d = > 0000161: 15 . > 0000162: e3 . > 0000163: a7 . > 0000164: b2 . > 0000165: fa . > 0000166: 14 . > 0000167: 4c L > 0000168: 0a . > 0000169: d2 . > 000016a: a1 . > 000016b: 7f . > 000016c: 6b k
It seems the difference is very minimal, I was expecting my terminal to be flooded with diff lines :)
follow-up: 40 comment:39 by , 10 years ago
Component: | avformat → undetermined |
---|---|
Keywords: | mpeg2video added; md5 mpeg2 ac3 mkv removed |
Resolution: | → duplicate |
Status: | new → closed |
$ ffmpeg -i cut.mkv -map v -map a -c:v copy -c:a copy -f md5 -
The expected output for this command line is 0b0257acc6662c99b71e879d27705ce9 (for current FFmpeg git head - b627385b - I suspect this could change in the future as it has changed - once - in the past).
As explained in ticket #3871, the output of above command line generally depends on the enabled decoders. If the mpeg2video decoder gets disabled, the output becomes 726d4f2a3d79e000e0bbde6397311bb2 and the console output changes:
Stream #0:0(eng): Video: mpeg2video (Main), yuv420p(tv), 720x576 [SAR 64:45 DAR 16:9], max. 7500 kb/s, 25 fps, 25 tbr, 1k tbn, 50 tbc (default)
Stream #0:0(eng): Video: mpeg2video, 720x576, max. 7500 kb/s, SAR 64:45 DAR 16:9, 25 fps, 25 tbr, 1k tbn, 50 tbc (default)
The console output change does not directly correspond to the md5 output change, it is just a more visible difference between the different configurations.
As explained in ticket #3871 both changes are not unexpected, demuxers can depend on decoders.
At least on all Linux systems, you can always disable the mpeg2video decoder with --disable-decoders --disable-hwaccels
, it makes no difference if you specify --disable-vdpau
, --disable-vaapi
or --disable-xvmc
or not. Your original configure line only contained --disable-decoders
but not --disable-hwaccels
. On your corei7 system at least one hardware acceleration library including its header is present (vdpau, vaapi or xvmc) meaning the mpeg2video decoder was enabled because it is required by all mpegvideo hwaccels, on the atom system none of the hardware acceleration libraries is present with an appropriate header meaning the mpeg2video decoder was not enabled. This explains the different output (for different binaries!) that you saw on different systems.
The md5 output changed with ac293b66 / 194be1f4 just as for ticket #3871.
If whatever you are trying to do makes sense for you, please compile one static binary that you share across systems. It took considerable time to solve this riddle and I don't think anybody has learned anything relevant from it.
follow-up: 41 comment:40 by , 10 years ago
Replying to cehoyos:
As explained in ticket #3871, the output of above command line generally depends on the enabled decoders. If the mpeg2video decoder gets disabled, the output becomes 726d4f2a3d79e000e0bbde6397311bb2 and the console output changes
You did'nt explain anything, you repeated several times that this is expected because libavformat depends on libavcodec, or because the build configuration is different. This is not an explanation.
Demuxing and decoding are different and independant things, and the former is supposed to be deterministic and reproducible.
Replying to cehoyos:
At least on all Linux systems, you can always disable the mpeg2video decoder with
--disable-decoders --disable-hwaccels
, it makes no difference if you specify--disable-vdpau
,--disable-vaapi
or--disable-xvmc
or not. Your original configure line only contained--disable-decoders
but not--disable-hwaccels
.
This is not what I foud out.
On my system, builds compiled with --disable-decoders --disable-hwaccels
or --disable-decoders --disable-vdpau --disable-vaapi
produce 0b0257acc6662c99b71e879d27705ce9, and builds with just --disable-decoders
produce 726d4f2a3d79e000e0bbde6397311bb2.
Builds with only --disable-decoders --disable-vaapi
or --disable-decoders --disable-vdpau
both produce 726d4f2a3d79e000e0bbde6397311bb2.
So the difference of MD5 is only present with builds with BOTH libvaapi and libvdpau.
Doesn't that look weird to you?
Replying to cehoyos:
The md5 output changed with ac293b66 / 194be1f4 just as for ticket #3871.
I did some testing, and at 8851437 (before the merge) MD5 is different from the reference, but always similar, with or without --disable-vdpau --disable-vaapi
. At 194be1f4, the MD5 was always 726d4f2a3d79e000e0bbde6397311bb2, at ac293b6 it started to differ with --disable-decoders --disable-vdpau --disable-vaapi
.
Given the fact that ac293b6 is supposed to be a simple merge of 194be1f4, don't you find that even more weird?
Replying to cehoyos:
On your corei7 system at least one hardware acceleration library including its header is present (vdpau, vaapi or xvmc) meaning the mpeg2video decoder was enabled because it is required by all mpegvideo hwaccels, on the atom system none of the hardware acceleration libraries is present with an appropriate header meaning the mpeg2video decoder was not enabled.
Yes, that is what I found out too. Even ignoring the fact that a decoder is compiled despite --disable-decoders
, it does not explain why it causes a different of MD5.
There is no decoder involved in demuxing, and there is no such thing as hardware accelerated demuxing.
This is only one of the things left unexplained: there is also the fact that the difference is there only when demuxing video and audio, and the 16 bytes difference among a several megabyte file.
I'd like to understand the results I get, and for now we only know that builds linked with libvaapi and libvdpau produce a different MD5.
I'm sincerely grateful for the time you have spent on this, but this is nowhere near a satisfying explanation of what is going on, and I don't think you should have closed this ticket.
comment:41 by , 10 years ago
Replying to ahthovaikied:
Replying to cehoyos:
At least on all Linux systems, you can always disable the mpeg2video decoder with
--disable-decoders --disable-hwaccels
, it makes no difference if you specify--disable-vdpau
,--disable-vaapi
or--disable-xvmc
or not. Your original configure line only contained--disable-decoders
but not--disable-hwaccels
.
This is not what I foud out.
On my system, builds compiled with--disable-decoders --disable-hwaccels
or--disable-decoders --disable-vdpau --disable-vaapi
produce 0b0257acc6662c99b71e879d27705ce9
You don't have the xvmc headers installed, meaning that --disable-decoders --disable-vdpau --disable-vaapi
disables the mpeg2video decoder while it is not disabled on my system with this configure line. If you use --disable-decoders --disable-hwaccels
, it makes no difference which headers you have installed, the mpeg2video decoder is never built.
follow-up: 43 comment:42 by , 10 years ago
Do I understand correctly that ac293b6 was a merge of 194be1f4 that is from Libav?
I checked and Libav 10.3 does produce consistent MD5 with the different configure options.
I went back to Libav 9.5 (1.5 year old), and not only is the MD5 still consistent with the different configure options, but it is also the same value as with v10.3.
I though both projects were technically similar, and the fork was just the result of social tensions among developers, it seems there is also technical differences. I'll give Libav a try.
follow-up: 44 comment:43 by , 10 years ago
Replying to ahthovaikied:
Do I understand correctly that ac293b6 was a merge of 194be1f4 that is from Libav?
I checked and Libav 10.3 does produce consistent MD5 with the different configure options.
I went back to Libav 9.5 (1.5 year old), and not only is the MD5 still consistent with the different configure options, but it is also the same value as with v10.3.
I though both projects were technically similar, and the fork was just the result of social tensions among developers, it seems there is also technical differences. I'll give Libav a try.
Assuming the change was introduced in 194be1f4, this is indeed a change from Libav which was merged in ac293b6. Libav 10.3 does not include this change AFAICT; you probably want to compare with Libav git/master, or simply any hash ≥ 194be1f4.
comment:44 by , 10 years ago
Replying to ubitux:
Assuming the change was introduced in 194be1f4, this is indeed a change from Libav which was merged in ac293b6. Libav 10.3 does not include this change AFAICT; you probably want to compare with Libav git/master, or simply any hash ≥ 194be1f4.
Thanks.
The problem is libav master does not build with --disable-decoders --disable-hwaccels
, I get link errors like:
libavcodec/libavcodec.a(vc1dsp.o): dans la fonction « ff_vc1dsp_init »: vc1dsp.c:(.text.unlikely+0x22d): référence indéfinie vers « ff_vc1dsp_init_x86 » collect2: error: ld returned 1 exit status make: *** [avprobe] Erreur 1
Same for v11_alpha1 or v11_beta1 (who have the commit).
I did a test at 194be1f4 (which builds successfully), and the MD5 are the same as for v9.5 or v10.3, with or without --disable-decoders --disable-hwaccels
, which seems more logical to me than FFmpeg's current behavior.
I have uploaded a 10s file (track_ticket_3902.mkv) on upload.ffmpeg.org cut from a problematic file.
The problem is still reproducible with this small file.