Wrong interpretation of (m)jpeg aspect ratio

[Ulf Zibis]

Summary of the bug:
How to reproduce:

ffmpeg -ss 04:45 -t 00:02 -i CYD_copy.vob -movflags +faststart -vf atadenoise CYD_atadenoise_4-45_%02d.jpg
ffmpeg version 3.4.4-0ubuntu0.18.04.1
built from Ubuntu distribution

The input video has 704x576 with [SAR 12:11 DAR 4:3].
The x/y dpi values of the extracted JPEGs then should be 11/12 instead 12/11.
The resulting JPEGs now have a DAR of 1.12 instead of 1.333 (4:3).

[Carl Eugen Hoyos]

This has absolutely no similarities with a valid ticket.

Please test current FFmpeg git head, attach a sample and provide a simplified (!) command line including complete, uncut console output that shows the issue.

[Ulf Zibis]

$ ffmpeg -i CYD_t-15.mp4 -ss 00:05 CYD.jpg
ffmpeg version 3.4.4-0ubuntu0.18.04.1 Copyright (c) 2000-2018 the FFmpeg developers
  built with gcc 7 (Ubuntu 7.3.0-16ubuntu3)
  configuration: --prefix=/usr --extra-version=0ubuntu0.18.04.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --enable-gpl --disable-stripping --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librubberband --enable-librsvg --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libopencv --enable-libx264 --enable-shared
  libavutil      55. 78.100 / 55. 78.100
  libavcodec     57.107.100 / 57.107.100
  libavformat    57. 83.100 / 57. 83.100
  libavdevice    57. 10.100 / 57. 10.100
  libavfilter     6.107.100 /  6.107.100
  libavresample   3.  7.  0 /  3.  7.  0
  libswscale      4.  8.100 /  4.  8.100
  libswresample   2.  9.100 /  2.  9.100
  libpostproc    54.  7.100 / 54.  7.100
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x555fc1a5b800] Could not find codec parameters for stream 0 (Video: h264 (avc1 / 0x31637661), none, 704x576, 831 kb/s): unspecified pixel format
Consider increasing the value for the 'analyzeduration' and 'probesize' options
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'CYD_t-15.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf57.83.100
  Duration: 00:00:15.00, start: 0.000000, bitrate: 655 kb/s
    Stream #0:0(und): Video: h264 (avc1 / 0x31637661), none, 704x576, 831 kb/s, SAR 12:11 DAR 4:3, 25 fps, 25 tbr, 12800 tbn, 25600 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(und): Audio: mp3 (mp4a / 0x6134706D), 48000 Hz, stereo, s16p, 114 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> mjpeg (native))
Press [q] to stop, [?] for help
[swscaler @ 0x555fc1b758e0] deprecated pixel format used, make sure you did set range correctly
Output #0, image2, to 'CYD.jpg':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf57.83.100
    Stream #0:0(und): Video: mjpeg, yuvj420p(pc), 704x576 [SAR 12:11 DAR 4:3], q=2-31, 200 kb/s, 25 fps, 25 tbn, 25 tbc (default)
    Metadata:
      handler_name    : VideoHandler
      encoder         : Lavc57.107.100 mjpeg
    Side data:
      cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: -1
[image2 @ 0x555fc1a5fea0] Could not get frame filename number 2 from pattern 'CYD.jpg' (either set updatefirst or use a pattern like %03d within the filename pattern)
av_interleaved_write_frame(): Invalid argument
[image2 @ 0x555fc1a5fea0] Could not get frame filename number 2 from pattern 'CYD.jpg' (either set updatefirst or use a pattern like %03d within the filename pattern)
av_interleaved_write_frame(): Invalid argument
[image2 @ 0x555fc1a5fea0] Could not get frame filename number 2 from pattern 'CYD.jpg' (either set updatefirst or use a pattern like %03d within the filename pattern)
av_interleaved_write_frame(): Invalid argument
[image2 @ 0x555fc1a5fea0] Could not get frame filename number 2 from pattern 'CYD.jpg' (either set updatefirst or use a pattern like %03d within the filename pattern)
av_interleaved_write_frame(): Invalid argument
[image2 @ 0x555fc1a5fea0] Could not get frame filename number 2 from pattern 'CYD.jpg' (either set updatefirst or use a pattern like %03d within the filename pattern)
av_interleaved_write_frame(): Invalid argument
[image2 @ 0x555fc1a5fea0] Could not get frame filename number 2 from pattern 'CYD.jpg' (either set updatefirst or use a pattern like %03d within the filename pattern)
av_interleaved_write_frame(): Invalid argument
[image2 @ 0x555fc1a5fea0] Could not get frame filename number 2 from pattern 'CYD.jpg' (either set updatefirst or use a pattern like %03d within the filename pattern)
av_interleaved_write_frame(): Invalid argument
[image2 @ 0x555fc1a5fea0] Could not get frame filename number 2 from pattern 'CYD.jpg' (either set updatefirst or use a pattern like %03d within the filename pattern)
av_interleaved_write_frame(): Invalid argument
[image2 @ 0x555fc1a5fea0] Could not get frame filename number 2 from pattern 'CYD.jpg' (either set updatefirst or use a pattern like %03d within the filename pattern)
av_interleaved_write_frame(): Invalid argument
[image2 @ 0x555fc1a5fea0] Could not get frame filename number 2 from pattern 'CYD.jpg' (either set updatefirst or use a pattern like %03d within the filename pattern)
av_interleaved_write_frame(): Invalid argument
[image2 @ 0x555fc1a5fea0] Could not get frame filename number 2 from pattern 'CYD.jpg' (either set updatefirst or use a pattern like %03d within the filename pattern)
av_interleaved_write_frame(): Invalid argument
[image2 @ 0x555fc1a5fea0] Could not get frame filename number 2 from pattern 'CYD.jpg' (either set updatefirst or use a pattern like %03d within the filename pattern)
av_interleaved_write_frame(): Invalid argument
frame=   13 fps=0.0 q=24.8 Lsize=N/A time=00:00:00.52 bitrate=N/A dup=12 drop=0 speed=4.22x    
video:405kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Conversion failed!

[Ulf Zibis]

There is "Conversion failed!", but the desired JPG was created, see attachments.

[llogan]

Two issues:

Your ffmpeg is old. Only recent builds from the git master branch are supported here due to the level of development that occurs. ​Download a build or compile.

You need to add -frames:v 1 to output a single frame. Otherwise use a pattern, such as output_%03d.jpg, as specified in the error message.

[Ulf Zibis]

$ ~/Projects/ffmpeg/ffmpeg-git-20181218-amd64-static/ffmpeg -i CYD_t-15.mp4 -ss 00:05 -frames:v 1 CYD_2.jpg
ffmpeg version N-47741-ga1f0dd24f6-static https://johnvansickle.com/ffmpeg/  Copyright (c) 2000-2018 the FFmpeg developers
  built with gcc 6.3.0 (Debian 6.3.0-18+deb9u1) 20170516
  configuration: --enable-gpl --enable-version3 --enable-static --disable-debug --disable-ffplay --disable-indev=sndio --disable-outdev=sndio --cc=gcc-6 --enable-fontconfig --enable-frei0r --enable-gnutls --enable-gray --enable-libaom --enable-libfribidi --enable-libass --enable-libvmaf --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librubberband --enable-libsoxr --enable-libspeex --enable-libvorbis --enable-libopus --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libdav1d --enable-libxvid --enable-libzvbi --enable-libzimg
  libavutil      56. 24.101 / 56. 24.101
  libavcodec     58. 42.102 / 58. 42.102
  libavformat    58. 24.101 / 58. 24.101
  libavdevice    58.  6.101 / 58.  6.101
  libavfilter     7. 46.101 /  7. 46.101
  libswscale      5.  4.100 /  5.  4.100
  libswresample   3.  4.100 /  3.  4.100
  libpostproc    55.  4.100 / 55.  4.100
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x5eceb00] Could not find codec parameters for stream 0 (Video: h264 (avc1 / 0x31637661), none, 704x576, 830 kb/s): unspecified pixel format
Consider increasing the value for the 'analyzeduration' and 'probesize' options
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'CYD_t-15.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf58.24.101
  Duration: 00:00:15.00, start: 0.000000, bitrate: 660 kb/s
    Stream #0:0(und): Video: h264 (avc1 / 0x31637661), none, 704x576, 830 kb/s, SAR 12:11 DAR 4:3, 25 fps, 25 tbr, 12800 tbn, 25600 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(und): Audio: mp3 (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 114 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> mjpeg (native))
Press [q] to stop, [?] for help
[swscaler @ 0x5f0e800] deprecated pixel format used, make sure you did set range correctly
Output #0, image2, to 'CYD_2.jpg':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf58.24.101
    Stream #0:0(und): Video: mjpeg, yuvj420p(pc), 704x576 [SAR 12:11 DAR 4:3], q=2-31, 200 kb/s, 25 fps, 25 tbn, 25 tbc (default)
    Metadata:
      handler_name    : VideoHandler
      encoder         : Lavc58.42.102 mjpeg
    Side data:
      cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: -1
frame=    1 fps=0.0 q=4.9 Lsize=N/A time=00:00:00.04 bitrate=N/A dup=1 drop=1 speed=0.167x    
video:39kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

[Gyan]

Not clear what the issue is. ffprobe shows the output JPG to have a SAR of 12:11 which results in a DAR of 4:3. ExifTool also shows the JPG to have XResolution 12 and YResolution 11.

[Ulf Zibis]

The unit of SAR is length/dot
The unit of x/y-resolution is dots/length
So values 12/11 have to be switched for dpi.
If you print the picture on paper, you will get:
x-width = 704/12 = 58.6666... inch
y-height = 576/11 = 52.3636... inch
The resulting DAR then is 1.12 instead 1.333 (4:3).

Additionally it may be better to set the dpi to a reasonable value for the case if one tries to print the pictures on paper. 72 dpi is a common value, so here 66/72 dpi may be practicable. For higher resolutions e.g. HD-Video, the print should not exceed a normal paper sheet.

[Gyan]

Which app are you checking this in? Photoshop CS5.1 shows me the output with a DAR of 4:3 (raster area is 768x576) when pixel aspect ratio correction is on, which is the desired result.

[Gyan]

As per the JPEG specification at ​https://www.w3.org/Graphics/JPEG/jfif3.pdf, (last tip on pg 7)

if you only want to specify a pixel aspect ratio, put 0 for the units field in the special APP0 field. Xdensity and Ydensity can then be programmed for the desired aspect ratio. Xdensity = 1, Ydensity = 1 will program a 1:1 aspect ratio. Xdensity and Ydensity should always be non-zero.

Also, see on pg. 5

units = 0: no units, X and Y specify the pixel aspect ratio
units = 1: X and Y are dots per inch
units = 2: X and Y are dots per cm

Our encoder always writes units as 0, so Irfanview is at fault here.

put_bits(p, 8, 0); /* units type: 0 - aspect ratio */

[Ulf Zibis]

I'm using IrfanView, see attachment.

I'm wondering about the interpretation from Photoshop.
DPI means "dots per inch", so with 12 dpi 704 pixel/dots make 58.6666... inch etc...
Maybe with "correction" at Photoshop a x/y switch is meant.

[Ulf Zibis]

Sorry for the last comment.
You are right, IrfanView is wrong.

[Carl Eugen Hoyos]

While the author of the specification did not do his best to clarify, doubts about his intentions are unneeded.

[Gyan]

I don't see any doubt of anyone's intentions in the comments above. What the (open) issue here?