#7203 closed defect (invalid)
Problem with encoding type "Cyrillic (DOS)" with metadata delivery.
Reported by: | max79 | Owned by: | |
---|---|---|---|
Priority: | normal | Component: | undetermined |
Version: | unspecified | Keywords: | id3v2 |
Cc: | Blocked By: | ||
Blocking: | Reproduced by developer: | no | |
Analyzed by developer: | no |
Description
Summary of the bug:
TITLE metadata tag should looks like this- "Евангелие о следовании за Христом" for my example below.
To reproduce:
ffmpeg version N-91024-g293a6e8332 Copyright (c) 2000-2018 the FFmpeg developers built with gcc 7.3.0 (GCC) configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-bzlib --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom --enable-libmfx --enable-amf --enable-ffnvcodec --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth libavutil 56. 18.100 / 56. 18.100 libavcodec 58. 19.101 / 58. 19.101 libavformat 58. 13.102 / 58. 13.102 libavdevice 58. 4.100 / 58. 4.100 libavfilter 7. 21.100 / 7. 21.100 libswscale 5. 2.100 / 5. 2.100 libswresample 3. 2.100 / 3. 2.100 libpostproc 55. 2.100 / 55. 2.100 [mp3 @ 0000000000498380] Skipping 313 bytes of junk at 4096. [mp3 @ 0000000000498380] Estimating duration from bitrate, this may be inaccurate Input #0, mp3, from 'https://audio3.azbyka.ru/Svjashhennoe_pisanie/nikola_serbski_tolk/01_Evangelie_o_sledovanii_za_Xristom.mp3': Metadata: title : ├Е├в├а├н├г├е├л├и├е ├о ├▒├л├е├д├о├в├а├н├и├и ├з├а ├Х├░├и├▒├▓├о├м genre : Other album : ├Б├е├▒├е├д├╗ ├н├а ├Е├в├а├н├г├е├л├и├┐ 1├╖. track : 1 artist : ├С├в├┐├▓├и├▓├е├л├╝ ├Н├и├к├о├л├а├й ├С├е├░├б├▒├к├и├й id3v2_priv.WM/MediaClassSecondaryID: \x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00 id3v2_priv.WM/MediaClassPrimaryID: \xbc}`\xd1#\xe3\xe2K\x86\xa1H\xa4*(D\x1e id3v2_priv.AverageLevel: {\x04\x00\x00 id3v2_priv.PeakValue: !\x00\x00\x00 Duration: 00:39:40.34, start: 0.000000, bitrate: 96 kb/s Stream #0:0: Audio: mp3, 44100 Hz, mono, fltp, 96 kb/s At least one output file must be specified
Attachments (1)
Change History (13)
comment:1 by , 6 years ago
Component: | ffmpeg → undetermined |
---|---|
Priority: | important → normal |
by , 6 years ago
Attachment: | 01-vstuplenie.mp3 added |
---|
Problem with encoding type "Cyrillic (DOS)" with metadata delivery.
comment:2 by , 6 years ago
Please see attachment. TITLE metadata tag from this example should looks like this:
"Дорожка 1"
but now it looks like:
title : ├Д├о├░├о├ж├к├а 1
Anyway you can download it from here: https://azbyka.ru/audio/audio1/Zhitija-i-tvorenija-svjatykh/Nikolaj-Serbskij/molytvy-na-ozere/01-vstuplenie.mp3
comment:3 by , 6 years ago
This file uses id3v2.3 tags. The TIT2-tag (the tag containing the title) is as follows in hex: 0x54 49 54 32 00 00 00 0A 00 00 00 C4 EE F0 EE E6 EA E0 20 31. According to the standard the 0x00 after the length field indicates that the tag uses ISO-8859-1 as encoding, an encoding that does not contain cyrillic characters. For such purposes Unicode could (and should) be used, but isn't. This is a bug in the tool that created said file, not in FFmpeg.
Btw: The last nine bytes are the actual titel; in Windows-1251 they would be read as "Дорожка 1"; in the Cyrillic DOS code page 866 that you are referring to it means "─юЁюцър 1". In ISO-8859-1 they mean "Äîðîæêà 1". FFmpeg's output to the console is encoded as UTF-8, but cmd.exe (that you seem to use) expects applications to use the native legacy codepage of the system (for Russian Windows versions, this is usually Code page 866; cmd.exe is by the way Unicode compatible). The UTF-8 that FFmpeg writes to the console is 0xC3 84 C3 AE C3 B0 C3 AE C3 A6 C3 AA C3 A0 20 31. In CP 866 0xC3 is "├" whereas 0x84 is "Д". That six of the seven characters of the word (seem to) have been preserved does not really have a deeper meaning. It is accidental.
comment:4 by , 6 years ago
Thank you for clarification. But if it's not ffmpeg bug why you didn't close this ticket?
follow-up: 7 comment:5 by , 6 years ago
I am not an FFmpeg developer, but just an interested visitor to this board. I don't know whether I am allowed to close tickets and therefore I leave it to the real guys in charge.
comment:6 by , 6 years ago
Keywords: | id3v2 added |
---|---|
Resolution: | → invalid |
Status: | new → closed |
comment:7 by , 6 years ago
Replying to mkver:
I am not an FFmpeg developer, but just an interested visitor to this board. I don't know whether I am allowed to close tickets and therefore I leave it to the real guys in charge.
Please go ahead and close tickets if you think they should be closed, and provide an explanation why it should be closed (like you did in comment:3). If someone disagrees they can always add a comment and/or re-open it.
comment:8 by , 5 years ago
Resolution: | invalid |
---|---|
Status: | closed → reopened |
So, you say this is not bug in ffmpeg. I believe you. But if russian characters are in PATH in windows the problem it the same! This IS A BUG! The problem is only on windows))
PS C:\Users\xxxx> L:\5777\ffmpeg-20190911-944d76a-win64-static\bin\ffprobe.exe "Z:\Басков Николай (Flac)\2003 - Сила любви (избранное)\Николай Басков - Сила любви (избранное).flac" ffprobe version N-94862-g944d76a3e0 Copyright (c) 2007-2019 the FFmpeg developers built with gcc 9.1.1 (GCC) 20190807 configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libdav1d --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom --enable-libmfx --enable-ffnvcodec --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth --enable-libopenmpt --enable-amf libavutil 56. 35.100 / 56. 35.100 libavcodec 58. 56.101 / 58. 56.101 libavformat 58. 32.104 / 58. 32.104 libavdevice 58. 9.100 / 58. 9.100 libavfilter 7. 58.102 / 7. 58.102 libswscale 5. 6.100 / 5. 6.100 libswresample 3. 6.100 / 3. 6.100 libpostproc 55. 6.100 / 55. 6.100 Input #0, flac, from 'Z:\╨С╨░╤Б╨║╨╛╨▓ ╨Э╨╕╨║╨╛╨╗╨░╨╣ (Flac)\2003 - ╨б╨╕╨╗╨░ ╨╗╤О╨▒╨▓╨╕ (╨╕╨╖╨▒╤А╨░╨╜╨╜╨╛╨╡)\╨Э╨╕╨║╨╛╨╗╨░╨╣ ╨С╨░╤Б╨║╨╛╨▓ - ╨б╨╕╨╗╨░ ╨╗╤О╨▒╨▓╨╕ (╨╕╨╖╨▒╤А╨░╨╜╨╜╨╛╨╡).flac': Metadata: ARTIST : ╨Э╨╕╨║╨╛╨╗╨░╨╣ ╨С╨░╤Б╨║╨╛╨▓ TITLE : O sole Mio ALBUM : ╨б╨╕╨╗╨░ ╨╗╤О╨▒╨▓╨╕ (╨╕╨╖╨▒╤А╨░╨╜╨╜╨╛╨╡) DATE : 2001 track : 02 GENRE : Vocal COMMENT : Track 2 Duration: 01:05:28.69, start: 0.000000, bitrate: 844 kb/s Stream #0:0: Audio: flac, 44100 Hz, stereo, s16
comment:9 by , 5 years ago
Wow!!! It is your problem after all even in Metadata section! Here what it looks like in linux (compare to prev. comment):
ffprobe ./Николай\ Басков\ -\ Сила\ любви\ \(избранное\).flac ffprobe version 4.1.4-1+b2 Copyright (c) 2007-2019 the FFmpeg developers built with gcc 9 (Debian 9.2.1-1) configuration: --prefix=/usr --extra-version=1+b2 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared libavutil 56. 22.100 / 56. 22.100 libavcodec 58. 35.100 / 58. 35.100 libavformat 58. 20.100 / 58. 20.100 libavdevice 58. 5.100 / 58. 5.100 libavfilter 7. 40.101 / 7. 40.101 libavresample 4. 0. 0 / 4. 0. 0 libswscale 5. 3.100 / 5. 3.100 libswresample 3. 3.100 / 3. 3.100 libpostproc 55. 3.100 / 55. 3.100 Input #0, flac, from '/mnt/hgfs/Eurovision Song Contest 2019. Tel Aviv, Israel/Николай Басков - Сила любви (избранное).flac': Metadata: ARTIST : Николай Басков TITLE : O sole Mio ALBUM : Сила любви (избранное) DATE : 2001 track : 02 GENRE : Vocal COMMENT : Track 2 Duration: 01:05:28.69, start: 0.000000, bitrate: 844 kb/s Stream #0:0: Audio: flac, 44100 Hz, stereo, s16
comment:10 by , 5 years ago
Status: | reopened → open |
---|
https://ffmpeg.zeranoe.com/forum/viewtopic.php?f=7&t=7520&p=16316&hilit=cyrillic#top
As you can see this problem is common!!
comment:11 by , 5 years ago
Resolution: | → invalid |
---|---|
Status: | open → closed |
This issue is about two things:
- A file that claims to use cp ISO-8859-1, but actually isn't (in the sense that when the bytes are interpreted via ISO-8859-1 the result is not what the user expected). As explained, the file in question shows the unexpected "Äîðîæêà 1" even when the shell expects UTF-8 input.
- cmd.exe showing an even weirder output than "Äîðîæêà 1" (and doing so even for valid files) because ffmpeg outputs UTF-8 whereas cmd.exe expects something different.
Your issue is lacking the first part and the second part is a duplicate of #5398. I therefore reclose this.
comment:12 by , 5 years ago
So, #5398 was fixed, https://azbyka.ru/audio/audio1/Zhitija-i-tvorenija-svjatykh/Nikolaj-Serbskij/molytvy-na-ozere/01-vstuplenie.mp3 now works correctly (except for
id3v2_priv.AverageLevel: {\x0f\x00\x00 id3v2_priv.WM/MediaClassPrimaryID: \xbc}`\xd1#\xe3\xe2K\x86\xa1H\xa4*(D\x1e id3v2_priv.WM/MediaClassSecondaryID: \x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00 id3v2_priv.WM/UniqueFileIdentifier: ;\x00\x00\x00 id3v2_priv.PeakValue: !f\x00\x00
and the attached file has as mkver correctly said
ffprobe version git-2020-02-27-9b22254 Copyright (c) 2007-2020 the FFmpeg developers built with gcc 9.2.1 (GCC) 20200122 configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libdav1d --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom --enable-libmfx --enable-ffnvcodec --enable-cuda-llvm --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth --enable-libopenmpt --enable-amf libavutil 56. 42.100 / 56. 42.100 libavcodec 58. 73.102 / 58. 73.102 libavformat 58. 39.101 / 58. 39.101 libavdevice 58. 9.103 / 58. 9.103 libavfilter 7. 77.100 / 7. 77.100 libswscale 5. 6.100 / 5. 6.100 libswresample 3. 6.100 / 3. 6.100 libpostproc 55. 6.100 / 55. 6.100 [mp3 @ 00000208b1eec240] Estimating duration from bitrate, this may be inaccurate Input #0, mp3, from 'C:\Users\ZAQU\Downloads\01-vstuplenie.mp3': Metadata: title : Äîðîæêà 1 album : Íåèçâåñòíûé äèñê (05.02.2009 10:58:36) track : 1 TLEN : 67240 id3v2_priv.WM/UniqueFileIdentifier: ;\x00\x00\x00 id3v2_priv.WM/MediaClassSecondaryID: \x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00 id3v2_priv.WM/MediaClassPrimaryID: \xbc}`\xd1#\xe3\xe2K\x86\xa1H\xa4*(D\x1e id3v2_priv.AverageLevel: {\x0f\x00\x00 id3v2_priv.PeakValue: !f\x00\x00 Duration: 00:01:07.04, start: 0.000000, bitrate: 128 kb/s Stream #0:0: Audio: mp3, 44100 Hz, stereo, fltp, 128 kb/s
Please provide an input sample.