Opened 3 years ago

Closed 3 years ago

Last modified 2 years ago

#9376 closed defect (invalid)

Wrong conversion from flat to equirectangular

Reported by: jumpjack Owned by:
Priority: normal Component: avfilter
Version: unspecified Keywords:
Cc: jumpjack Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

Summary of the bug:
reprojection between flat and equirectangular apparently add an undesired rotation to the image (roll != 0)

ffmpeg -hide_banner   -i test.png          								-vf v360=fisheye:e                      -y test-equi.png
ffmpeg -hide_banner   -i test-equi.png    									-vf v360=e:e:yaw=10:pitch=10   					-y test-equi-rotated.png
ffmpeg -hide_banner   -i test-equi-rotated.png   					-vf v360=e:flat   											-y test-flat.png
ffmpeg -hide_banner   -i test-flat.png     								-vf v360=flat:e,colorkey=0xFFFFFF:0.1  	-y test-overlay-rotated.png
ffmpeg -hide_banner   -i test-overlay-rotated.png    			-vf v360=e:e:pitch=-10:yaw=-10    			-y test-overlay-back.png
ffmpeg -i test-equi.png -i test-overlay-back.png -filter_complex overlay 						-y test-final.png

Last image test-final.png should appear identical to test-equi.png, but it is not.

Version:
ffmpeg version 2021-07-06-git-758e2da289-full_build-www.gyan.dev Copyright (c) 2000-2021 the FFmpeg developers

built with gcc 10.3.0 (Rev2, Built by MSYS2 project)
configuration: --enable-gpl --enable-version3 --enable-static --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-lzma --enable-libsnappy --enable-zlib --enable-librist --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enable-libcaca --enable-sdl2 --enable-libdav1d --enable-libzvbi --enable-librav1e --enable-libsvtav1 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-libaom --enable-libopenjpeg --enable-libvpx --enable-libass --enable-frei0r --enable-libfreetype --enable-libfribidi --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-libglslang --enable-vulkan --enable-opencl --enable-libcdio --enable-libgme --enable-libmodplug --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libshine -- libavutil 57. 0.100 / 57. 0.100
libavcodec 59. 3.101 / 59. 3.101
libavformat 59. 4.100 / 59. 4.100
libavdevice 59. 0.100 / 59. 0.100
libavfilter 8. 0.103 / 8. 0.103
libswscale 6. 0.100 / 6. 0.100
libswresample 4. 0.100 / 4. 0.100
libpostproc 56. 0.100 / 56. 0.100

Attachments (2)

test_equi-test-mini.jpg (25.5 KB ) - added by jumpjack 3 years ago.
initial
test_merge-test-mini.jpg (25.2 KB ) - added by jumpjack 3 years ago.
final

Download all attachments as: .zip

Change History (16)

comment:1 by Michael Koch, 3 years ago

Can't you show the problem with a simpler example? Is colorkey and overlay really required for reproducing the problem?
One thing I saw is that in the 5th command you swapped the order of yaw and pitch. That doesn't work as you might expect. The default rotation order is yaw-pitch-roll, regardless of the order in the command line. Use the "rorder" option of the v360 filter.

Michael

comment:2 by Michael Koch, 3 years ago

When converting to "flat" format it's a good idea to set the horizontal and vertical field of view (h_fov and v_fov). And when using "flat" as input, set the input field of view (ih_fov and iv_fov).

Michael

comment:3 by Michael Koch, 3 years ago

Resolution: fixed
Status: newclosed

Add :rorder=pyr to the 5th command line, then it works.

Michael

by jumpjack, 3 years ago

Attachment: test_equi-test-mini.jpg added

initial

by jumpjack, 3 years ago

Attachment: test_merge-test-mini.jpg added

final

comment:4 by jumpjack, 3 years ago

Resolution: fixed
Status: closedreopened

Ok try this sequence:

ffmpeg -hide_banner   -i .\test.png                         -vf v360=fisheye:e                      -y .\test_equi-test.png
ffmpeg -hide_banner   -i .\test_equi-test.png               -vf v360=e:e:yaw=10:pitch=10          -y .\test_equi_rotated-test.png
ffmpeg -hide_banner   -i .\test_equi_rotated-test.png       -vf v360=e:flat                         -y .\test_crop_flat-test.png
ffmpeg -hide_banner   -i .\test_crop_flat-test.png          -vf v360=flat:e,colorkey=0xFFFFFF:0.1   -y .\test_crop_equi-test.png
@set /p=Hit ENTER to continue...
ffmpeg -hide_banner   -i .\test_equi_rotated-test.png -i  .\test_crop_equi-test.png -filter_complex overlay -y .\test_merge_rotated-test.png
ffmpeg -hide_banner   -i .\test_merge_rotated-test.png      -vf v360=e:e:yaw=-10:pitch=-10            -y .\test_merge-test.png

I added the pause because the colorkey is not working as I want, for each image the fill-in color is chosen randomly and I don't know how to determine it, so I manually re-save the png overlay with the right transparent color.

I attached the initial equirectangular image and final result.

in reply to:  1 comment:5 by jumpjack, 3 years ago

Replying to Michael Koch:

Can't you show the problem with a simpler example? Is colorkey and overlay really required for reproducing the problem?

I don't understand when/where/why the problem occurs, so I included my whole sequence of commands.

comment:6 by Michael Koch, 3 years ago

Resolution: invalid
Status: reopenedclosed

This is the bug tracker and in your case it's the wrong place to ask. Because you aren't reporting a bug in FFmpeg. I'm pretty sure it's only a problem in your command line. Please ask in the FFmpeg-user list and describe exactly what you are trying to do and what doesn't work. Please also try to show the problem with fewer FFmpeg commands.

Michael

Last edited 3 years ago by Michael Koch (previous) (diff)

comment:7 by jumpjack, 3 years ago

It's a bug, please try the script.
And it does not depend on order of yaw and pitch.

My script does this:
1) Fisheye --> equirectangular
2) Rotate
3) Extract cropping in flat projection
4) Flat cropping --> equirectangular cropping
(user edit)
5) Overlay equirectangular cropping to original equirectangular image
6) Rotate back the new equirectangular image to original position ==> BUG! A 2.1 degrees roll is added to the rotation

But if you don't want to fix it is does not matter, the image is still there and the subject is unmodified (apart from user intervention), it's just erroneously rotated, hence it is a minor bug.

comment:8 by Michael Koch, 3 years ago

You are still applying the back-rotations in the wrong order. You must add :rorder=pyr if you want first pitch and then yaw rotation.

Michael

in reply to:  8 comment:9 by jumpjack, 3 years ago

Replying to Michael Koch:

You are still applying the back-rotations in the wrong order. You must add :rorder=pyr if you want first pitch and then yaw rotation.

If FFMpeg intentionally ignores what I write, we have another bug.
But also applying your trick, it does not work (of course: it does not matter the order of parameters, if I use the same order in the two lines).

Anyway as I said, nevermind, I'll live with it.

comment:10 by Michael Koch, 3 years ago

I did test it and with :rorder=pyr it worked perfectly. Without this modification I saw the same small roll rotation as you did.

First rotation:
v360=e:e:yaw=10:pitch=10 (rorder=ypr is unnecessary here, because it's the default)

Second (back-)rotation:
v360=e:e:yaw=-10:pitch=-10:rorder=pyr

Michael

comment:11 by jumpjack, 3 years ago

Resolution: invalidwontfix

This looks like a double bug to me:

  • FFmpeg ignores the keywords "pitch" and "yaw" and do what it likes with the numbers
  • FFMpeg introduces a roll upon changing only yaw and pitch; the order of the rotations should not matter, yaw=10:pitch=10 and pitch=10:yaw=10 should give same rotation, without adding any roll.

But if you like as it is, leave it as it is.
I mark the issue as "wontfix".

in reply to:  11 comment:12 by Michael Koch, 3 years ago

Replying to jumpjack:

This looks like a double bug to me:

  • FFmpeg ignores the keywords "pitch" and "yaw" and do what it likes with the numbers

The v360 filter does exactly what it's supposed to do.

  • FFMpeg introduces a roll upon changing only yaw and pitch; the order of the rotations should not matter, yaw=10:pitch=10 and pitch=10:yaw=10 should give same rotation, without adding any roll.

Sorry, that's wrong. We are talking about a spherical coordinate system, like the geographical coordinates on earth.
Case 1: You walk 1km north, then turn 90° left and walk 1km to the west.
Case 2: You walk 1km to the west, then turn 90° right and walk 1km north.
You might think that in both cases you arrive at the same place. But that's wrong.
It becomes clear when you walk longer distances:
Case 3: You start at the equator in Africa and walk north until you reach the north pole, then turn 90° left and walk the same distance. You will arrive somewhere in central America.
Case 4: You start at the equator in Africa and walk west until you are in central America, then turn 90° right and walk the same distance. You will arrive at the north pole.
That means the order of rotations is important.
When you have made yaw and pitch rotations (in this order) and want to undo these rotations, you must undo them in reverse order. First pitch and then yaw.

Michael

comment:13 by Elon Musk, 3 years ago

Resolution: wontfixinvalid

comment:14 by Carl Eugen Hoyos, 2 years ago

Version: 4.3.2unspecified
Note: See TracTickets for help on using tickets.