Opened 4 months ago
Last modified 3 months ago
#11080 new task
FFmpeg timestamps do not consistently agree with packet timestamps
| Reported by: | markfilipak | Owned by: | |
|---|---|---|---|
| Priority: | normal | Component: | documentation |
| Version: | unspecified | Keywords: | |
| Cc: | MasterQuestionable | Blocked By: | |
| Blocking: | Reproduced by developer: | no | |
| Analyzed by developer: | no |
Description (last modified by )
Summary of the bug:
I made a 4 second clip having 97 actual frames. Physical order is DTS.
VLC and PowerDVD play the clip perfectly.
'-f framecrc' & packet analyzer agree: 97 frames, same DTSes & PTSes.
'-vf showinfo' disagrees: 53 frames, a gap, and duplicates.
'-show_frames' disagrees: 53 frames, a different gap, and different duplicates.
'-vf showinfo' & '-show_frames' disagree substantially.
Other effects:
MPV exhibits a 2 second glitch that matches the gap.
'-ss' and '-to' exhibit the same sorts of problems.
__packet analyzer__ _____framecrc______ ___showinfo___ ______show_frames_______
(physical order) (DTS order) (DTS order) (DTS order)
___DTS___ ___PTS___ ___DTS___ ___PTS___ PN ___PTS___ DN ___DTS___ ___PTS___
504133642 504137396 504133642 504137396 0 504137396 I
504137396 504148657 504137396 504148657 3 504148657 P
504141150 504141150 504141150 1 504141150 all these PTSes = DTSes - 3 frames
504144903 504144903 504144903 2 504144903 ¦
504148657 504156165 504148657 504156165 5 504156165 P 0 504148657 504137396 I
504152411 504152411 504152411 4 504152411 1 504152411 504141150
504156165 504167426 504156165 504167426 8 504167426 P 2 504156165 504144903
504159918 504159918 504159918 6 504159918 3 504159918 504148657 P
504163672 504163672 504163672 7 504163672 4 504163672 504152411
504167426 504174933 504167426 504174933 10 504174933 P 5 504167426 504156165 P
504171180 504171180 504171180 9 504171180 6 504171180 504159918
504174933 504186195 504174933 504186195 13 504186195 P 7 504174933 504163672
504178687 504178687 504178687 11 504178687 8 504178687 504167426 P?
504182441 504182441 504182441 12 504182441 9 504182441 504171180
504186195 504197456 504186195 504197456 16 504197456 P 10 504186195 504174933 P
504189948 504189948 504189948 14 504189948 11 504189948 504178687
504193702 504193702 504193702 15 504193702 12 504193702 504182441
===== MPV time pauses here momentarily, then jumps 3 frames later =====
504197456 504204963 504197456 504204963 18 504204963 P 13 504197456 504186195 P
504201210 504201210 504201210 17 504201210 14 504201210 504189948
504204963 504216225 504204963 504216225 15 504204963 504193702
===== MPV time jumps from here =====
504208717 504208717 504208717 19 504208717 16 504208717 504197456 P?
504212471 504212471 504212471 20 504212471 17 504212471 504201210
504216225 504223732 504216225 504223732 18 504216225 504204963 P
504219978 504219978 504219978 19 504219978 504208717
===================== SPLICE HERE ======================
504223731 504227485 504223731 504227485
504227485 504234993 504227485 504234993
504231239 504231239 504231239
504234993 504246254 504234993 504246254
504238746 504238746 504238746
504242500 504242500 504242500
504246254 504257515 504246254 504257515
504250008 504250008 504250008
504253761 504253761 504253761
504257515 504265023 504257515 504265023
504261269 504261269 504261269
504265023 504276284 504265023 504276284
504268776 504268776 504268776
504272530 504272530 504272530
504276284 504287545 504276284 504287545
504280038 504280038 504280038
504283791 504283791 504283791
504287545 504295053 504287545 504295053
504291299 504291299 504291299
504295053 504306314 504295053 504306314
504298806 504298806 504298806
504302560 504302560 504302560
504306314 504317575 504306314 504317575
504310068 504310068 504310068
504313821 504313821 504313821
504317575 504325083 504317575 504325083
504321329 504321329 504321329
504325083 504332590 504325083 504332590
504328836 504328836 504328836
504332590 504340098 504332590 504340098
504336344 504336344 504336344
504340098 504347605 504340098 504347605
504343851 504343851 504343851
504347605 504355113 504347605 504355113
504351359 504351359 504351359
504355113 504362620 504355113 504362620
504358866 504358866 504358866
504362620 504370128 504362620 504370128
504366374 504366374 504366374
504370128 504377635 504370128 504377635
504373881 504373881 504373881
504377635 504385143 504377635 504385143
504381389 504381389 504381389
504385143 504396404 504385143 504396404 22 504396404 P 20 504385143 504212471
===== MPV time jumps to here =====
504388896 504388896 504388896
504392650 504392650 504392650 21 504392650 21 504392650 504392650 --+ best_effort_timestamp
504396404 504407665 504396404 504407665 25 504407665 22 504396404 504216225 P? ¦ switches
504400158 504400158 504400158 23 504400158 P 23 504400158 504396404 P ¦ from
504403911 504403911 504403911 24 504403911 24 504403911 504219978 ¦ 'pts'
504407665 504418926 504407665 504418926 28 504418926 I 25 504407665 504400158 ¦ to
504411419 504411419 504411419 26 504411419 I? 26 504411419 504223732 I ¦ 'pkt_dts'
504415173 504415173 504415173 27 504415173 27 504415173 504403911 ¦ here
504418926 504426434 504418926 504426434 30 504426434 28 504418926 504407665 I ¦
504422680 504422680 504422680 29 504422680 29 504422680 504411419 ¦
504426434 504433941 504426434 504433941 32 504433941 30 504426434 504415173 ¦
504430188 504430188 504430188 31 504430188 P 31 504430188 504418926 P ¦
504433941 504445203 504433941 504445203 35 504445203 P 32 504433941 504422680 ¦
504437695 504437695 504437695 33 504437695 P? 33 504437695 504426434 P ¦
504441449 504441449 504441449 34 504441449 34 504441449 504430188 ¦
504445203 504456464 504445203 504456464 38 504456464 P 35 504445203 504433941 P ¦
504448956 504448956 504448956 36 504448956 36 504448956 504437695 ¦
504452710 504452710 504452710 37 504452710 37 504452710 504441449 ¦
504456464 504467725 504456464 504467725 41 504467725 P 38 504456464 504445203 P ¦
504460218 504460218 504460218 39 504460218 39 504460218 504448956 ¦
504463971 504463971 504463971 40 504463971 40 504463971 504452710 ¦
504467725 504475233 504467725 504475233 43 504475233 41 504467725 504456464 P ¦
504471479 504471479 504471479 42 504471479 42 504471479 504460218 ¦
504475233 504486494 504475233 504486494 46 504486494 P 43 504475233 504463971 ¦
504478986 504478986 504478986 44 504478986 P? 44 504478986 504467725 P? ¦
504482740 504482740 504482740 45 504482740 45 504482740 504471479 ¦
504486494 504497755 504486494 504497755 52 504497755 I 46 504486494 504475233 P? ¦
504490248 504490248 504490248 47 504490248 47 504490248 504478986 ¦
504494001 504494001 504494001 48 504494001 48 504494001 504482740 --+
duplicate of 46 -> 49 504486494 P 49 "N/A" 504486494 P
duplicate of 47 -> 50 504490248 50 "N/A" 504490248
duplicate of 48 -> 51 504494001 51 "N/A" 504494001
52 "N/A" 504497755 I
How to reproduce:
ffmpeg -i Ticket_11080.m2ts -map 0 -copyts -c copy -dn -sn -f framecrc - >Ticket_11080.m2ts_framecrc.txt
ffmpeg -copyts -i Ticket_11080.m2ts -map 0:v -vf showinfo -c:v rawvideo -f null -muxdelay 0 - 2>Ticket_11080.m2ts_showinfo.txt
ffprobe -sexagesimal -select_streams v -show_frames -of flat -i Ticket_11080.m2ts >Ticket_11080.m2ts_show_frames.txt
[note]
ffmpeg version 2024-05-20-git-127ded5078-full_build-www.gyan.dev
[note] show_frames complains:
[null @ 000000000488bc40] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 134397 >= 134395
[null @ 000000000488bc40] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 134397 >= 134396
[null @ 000000000488bc40] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 134397 >= 134397
What 134397 is, is unknown. It is not a DTS or PTS or packet number (packet numbers are 0..73599).
Change History (92)
comment:1 by , 4 months ago
| Description: | modified (diff) |
|---|---|
| Summary: | FFmpeg timestamps do not consistenly agree with packet timestamps → FFmpeg timestamps do not consistently agree with packet timestamps |
comment:2 by , 4 months ago
comment:3 by , 4 months ago
I don't know where to upload Ticket_11080.m2ts.
Never mind. I found it in the FFmpeg docs.
follow-up: 5 comment:4 by , 4 months ago
https://streams.videolan.org/upload/
Select ffmpeg, input ticket, upload.
Sample should then be here https://samples.ffmpeg.org/ffmpeg-bugs/trac/ticket11080
comment:6 by , 4 months ago
| Description: | modified (diff) |
|---|
comment:8 by , 4 months ago
Replying to Balling:
It will when you will upload the file.
I did, and got "Upload complete". I'll try it again.
follow-up: 11 comment:10 by , 4 months ago
If you can't get it to upload to videolan - Upload it to a free 3rd party hosting site (eg. google drive, sendspace, mediafire, dropbox, workupload etc...), post the url, and a developer will mirror it on ffmpeg trac / videolan for this ticket
comment:11 by , 4 months ago
comment:12 by , 4 months ago
By the way, earlier today I sent the video to Fereidoon Khosravi, SVP of Market Development for Venera. If he follows through, Venera technicians are going to validate it with Venera Pulsar. He also promised to give me a license for Pulsar. I wouldn't count on either actually happening. We shall see.
follow-ups: 16 17 18 19 comment:13 by , 4 months ago
You moved the first two B frames (their display), so that made decoding of GOP impossible so it prints "co located POCs unavailable" right on the 2nd frame (and the frame has artefacts), try to use mpv on your file and step forward 1 frame with . button. Well, and also this again starts with non-IDR I frame.
Again, Open GOP looks like this:
https://forum.videohelp.com/images/imgfiles/Df6d4pR.jpg
Open GOP needs to start with two B frames, yours starts with an I frame that is wrong.
follow-ups: 20 29 comment:14 by , 4 months ago
Ticket_11080.m2ts is the same problem as ticket #11055 - not a valid stream
Not cut/appended on IDR boundaries, no IDR pictures , decreasing frame_num and decreasing POC violations in coded stream order
eg.
Notice frame_num drops from 243 to 191
There is a free online viewer; you can demux to elementary stream and upload
VTCLab Media Analyzer
https://media-analyzer.pro/app
hex
00002f7139 frame_num 243 (0xF3)
00003938a0 frame_num 191 (0xBF)
If you examine a retail open GOP BD, the frame_num increases with each coded picture including non-IDR "i" frames, until the next IDR where it resets to zero as per the h264 specs .
"The value of frame_num is constrained as follows:
– If the current picture is an IDR picture, frame_num shall be equal to 0."
When you stream copy & append segment the slice header frame_num , POC entries are retained from original stream. In Ticket_11080.m2ts, the latter appended section is closer in coded order to it's own IDR picture, so has a lower frame_num. When you join, there is now a decrease in frame_num - this is a spec violation
Either that section from <IDR> to the <frame before next IDR> from your original source stream has to be re-encoded before appending stream copied segments - so the values frame_num and POC values reset and follow specs (then you can also get frame accurate cuts) - this is what "smart rendering" software does - or you live with IDR boundaries and "coarse" edits (some open GOP BD's can have hundreds of frames before the next IDR)
follow-up: 34 comment:15 by , 4 months ago
This stream is even worse, pdr0, as it has PTS of zero on I frame, but two first B frames should have smaller PTS, as they are presented before I frame and since they cannot be fully decoded actually are just dropped, and are later used for decoding of another, third, B frame. LOL
Also VLC really fails with this stream it has horrible artefacts.
Also, why is 42 actual presented frame is frame 10 in encoded mode (lets say 10/42) and that B frame depends on P frame 6/8 and P frame 9/45? That is crazy.
comment:16 by , 4 months ago
I'm responding in the order your comments are in...
Replying to Balling:
You moved the first two B frames (their display),
I didn't move the first 2 B-frames, I dropped them because they are in the previous GOP. Here's the script:
: Cutting out a 4 second clip at the splice +/- 2 seconds. : : source is 00305+00306.m2ts : CUT1 = 1:32:49.814 = 5569.814 sec //running time to starting I-frame : CUT2 = 1:32:53.818 = 5573.818 sec //running time to ending I-frame : : CUT1: : (5569.814 sec)*(24/1.001 frames/sec) = 133542 frames : (2854113 ticks)+(133542 frames)*(3753.75 ticks/frame) = 504137395.5 ticks : = 504137396 ticks : PTS 504137396 is an I-frame in packet 75943446 : : step 1: ..________drop________ : \ : PTS order ..P B B I.. : ___________________________/ : / : DTS order I B B P.. : \ \ \ \ : 504126135 504129888 504133642 504137396 set CUT1=lt(pts\,504137396) : step 2: rewrite the I-frame's DTS : : PTS order I.. : _______/ : / : DTS order I P.. : \ \ : 504133642 504137396 set IDR=if(eq(DTS\,504126135)\,504133642,DTS) : CUT2: : (5573.818 sec)*(24/1.001 frames/sec) = 133638 frames : (2854113 ticks)+(133638 frames)*(3753.75 ticks/frame) = 504497755.5 ticks : = 504497755 ticks : PTS 504497755 is an I-frame in packet 76021654 : : PTS order ..P B B I : ___________________________/ ________drop________.. : / / : DTS order ..I B B P B B.. : \ \ \ \ : 504486494 504490248 504494001 504497755 set CUT2=gt(pts\,504497755) ffmpeg -copyts -i "c:\00305+00306.m2ts" -map 0 -bsf noise=drop='%CUT1%+%CUT2%',setts=dts='%IDR%':pts=PTS -c copy -sn -dn -muxdelay 0 "c:\Ticket_11080.m2ts" pause
Oh, look. DVBinspector is getting the running time wrong:
"pts: 0x1E0C86B4 (504137396) => 1:33:21.5266"
should be:
504137396-2854113 = (501283283 ticks)/(90000 ticks/sec) => 1:32:49.8142[5..]
DVBinspector isn't subtracting the initial PTS. The initial PTS has no relevance to actual frames.
I will comment on the rest in my next comment.
comment:17 by , 4 months ago
Replying to Balling:
You moved the first two B frames (their display) [refuted in the previous comment], so that made decoding of GOP impossible so it prints "co located POCs unavailable"
Only 'ffprobe -show_frames' says "co located POCs unavailable". I think that's because the bug is affecting decoding. 'ffmpeg -f framecrc' does not decode, so it's unaffected by the bug. That's why 'ffmpeg -f framecrc' gives correct timestamps that match the actual packets.
Look at the actual packets, please.
Look, I know it is hard for you to accept that FFmpeg's timestamps are wrong. You're using FFmpeg functions to 'prove' things. I'm using the actual packets. That's physical evidence.
You think the 2 B-frames are part of the right-side GOP. I think they're part of the left-side GOP that gets cut off. Let me ask you this: Why would dropping the 2 B-frames make any difference at all? Whether they're part of the left-side GOP or the right-side GOP, why would dropping them make a difference?
I think this is a very old bug.
follow-up: 23 comment:18 by , 4 months ago
Replying to Balling:
... Well, and also this again starts with non-IDR I frame.
I'm making the start frame (i.e., the I-frame) IDR. How can it be otherwise? It's the first frame!
comment:19 by , 4 months ago
Again, Open GOP looks like this:
https://forum.videohelp.com/images/imgfiles/Df6d4pR.jpg
What is that? Is it authoritative?
Open GOP needs to start with two B frames, yours starts with an I frame that is wrong.
H.222, Quote: "After a GOP, the first picture shall be an I-picture". I believe it means "After a GOP header". I've never seen a GOP header that was followed by a B-frame in the same packet.
follow-up: 25 comment:20 by , 4 months ago
Replying to pdr0:
Ticket_11080.m2ts is the same problem as ticket #11055 - not a valid stream
Not cut/appended on IDR boundaries, no IDR pictures , decreasing frame_num and decreasing POC violations in coded stream order
eg.
Notice frame_num drops from 243 to 191
The clip has only 97 frames.
Notice that frame_num applies to slices. Notice that H.264 says what to do with frame_num, but it never defines "frame_num".
comment:21 by , 4 months ago
making the start frame (i.e., the I-frame) IDR. How can it be otherwise? It's the first frame!
IDR frames are nal_unit_type 5. Yours is 1.
comment:22 by , 4 months ago
H.222, Quote: "After a GOP, the first picture shall be an I-picture"
That is true for H.222. Your file is H.264.
comment:23 by , 4 months ago
Replying to markfilipak:
Replying to Balling:
... Well, and also this again starts with non-IDR I frame.
I'm making the start frame (i.e., the I-frame) IDR. How can it be otherwise? It's the first frame!
You cannot change the slice/frame type without re-encoding .
If it was IDR to begin with, frame_num would be "0". The first slice has a frame_num of 227 - hence it's a non-IDR slice. That alone makes it a non valid codec video sequence as per the ITU h.264 specs
G.3.18
"The first picture of each coded video sequence in decoding order is an IDR picture."
7.4.3
"The value of frame_num is constrained as follows:
– If the current picture is an IDR picture, frame_num shall be equal to 0."
follow-up: 26 comment:24 by , 4 months ago
His source is Blu-ray, you cannot just quote the standard like this, it may be encoded using non-IDR frames throughout. Anyway, I think I need to show at least some proof: here Elecard 2023 shows that your first slice is actually non-IDR I frame. https://i.imgur.com/0V2RiWG.png (see it says nal_unit_type 1 (slice_layer_without_partitioning()))
The spec Table 7-1 says this is Coded slice of a non-IDR picture.
follow-up: 32 comment:25 by , 4 months ago
Replying to markfilipak:
Replying to pdr0:
Ticket_11080.m2ts is the same problem as ticket #11055 - not a valid stream
Not cut/appended on IDR boundaries, no IDR pictures , decreasing frame_num and decreasing POC violations in coded stream order
eg.
Notice frame_num drops from 243 to 191
The clip has only 97 frames.
Notice that frame_num applies to slices. Notice that H.264 says what to do with frame_num, but it never defines "frame_num".
Yes, if you look at the parser there are 4 slices per frame (as per the BD specs) . The frame_num is wrong because it was improperly cut from a longer segment with more frames (slices), as described in ticket 11055 . Because there are no IDR frames or slices, there is no reset to frame_num and you get discontinunities
"frame_num is used as an identifier for pictures "
https://patents.google.com/patent/US20140064363A1/en
" For each reference picture in H.264, there is a codeword frame_num that acts as a label for the reference picture. The frame_num indicates the decoding order and the frame_num must increase by 1 for each reference picture in decoding order otherwise the bitstream is not compliant to the standard."
comment:26 by , 4 months ago
Replying to Balling:
His source is Blu-ray, you cannot just quote the standard like this, it may be encoded using non-IDR frames throughout. Anyway, I think I need to show at least some proof: here Elecard 2023 shows that your first slice is actually non-IDR I frame. https://i.imgur.com/0V2RiWG.png (see it says nal_unit_type 1 (slice_layer_without_partitioning()))
Yes, but "encoded using non-IDR frames throughout" means there are IDR frames throughout as well . It is encoded with non-IDR throughout - The problem as mentioned before, and again in ticket 11055 is cuts/appends along non IDR boundaries
And the free analyzer I linked to above confirms the same thing as Elecard
https://media-analyzer.pro/app
nal_unit_type 1, 'Non-IDR Slice'
frame_num 227 (0xE3)
follow-up: 30 comment:27 by , 4 months ago
means there are IDR frames throughout as well
No. It means there are none or maybe only in the beginning. You need to guess which frames are actually recovery.
comment:28 by , 4 months ago
I dropped them because they are in the previous GOP.
No, the GOP starts with 2 B frames in coding order, they cannot be fully decoded, decode corrupt, which you can still see with ffmpeg -flags2 +showall -i "Ticket 11055.m2ts" cdasca.png
They are needed to decode frames after I frame. You cannot drop them.
comment:29 by , 4 months ago
comment:30 by , 4 months ago
Replying to Balling:
means there are IDR frames throughout as well
No. It means there are none or maybe only in the beginning. You need to guess which frames are actually recovery.
No , there are always many IDR frames in retail blu-ray
IDR frames are present at every chapter point at the miniumum. Have you ever looked at a retail blu-ray ?
As I mentioned earlier, some BD's have 100's of frames before the next IDR, but IDR's are always interspersed between non-IDR .
And you don't need to "guess" anything - that why I mentioned using stream analyzers such as Elecard in ticket 11055
comment:31 by , 4 months ago
@markfilipak Well, please read this, I think proof about Open GOP starting with two B frames is proof enough. https://forum.doom9.org/showthread.php?t=182920 and
https://forum.doom9.org/showthread.php?t=185523
There are examples, where I frames follow other I frames... I mean clearly it is not as simple as in some papers. In some papers though it is mentioned that scene change can insert I frame.
follow-up: 36 comment:32 by , 4 months ago
Replying to pdr0:
Replying to markfilipak:
Replying to pdr0:
Ticket_11080.m2ts is the same problem as ticket #11055 - not a valid stream
Not cut/appended on IDR boundaries, no IDR pictures , decreasing frame_num and decreasing POC violations in coded stream order
eg.
Notice frame_num drops from 243 to 191
The clip has only 97 frames.
Notice that frame_num applies to slices. Notice that H.264 says what to do with frame_num, but it never defines "frame_num".
Yes, if you look at the parser ...
You have a parser for MPEG2TS? Actual physical bits would help me to see what you're writing about. I write that because it seems to me you (or MPEG) are mixing stream frames, which have only timestamps and not frame numbers, with something that goes by frame numbers. I suspect they're actually frame offsets, not frame numbers. I suspect they're used for transforming slices between reference slices in other frames as opposed to transforming while frames. I suspect this has something to do with POC, which I find to be 'prescribed' rather than 'described', and so, seems rather vague and subject to interpretation.
Two things:
1 - If you have a parser for MPEG2TS, what is it? How do I get it? I can parse TS (as in MOV files) but not MPEG2TS.
2 - I have a proposal: Since this is going beyond a bug report and the time you are putting in is growing, how about I pay you? Money for instruction. That would only be fair to you and I'm willing to do it. If such instruction does reveal an actual bug, wonderful. If it doesn't that's fine too.
comment:33 by , 4 months ago
Here is elecard 2023 basic. And what he is talking about is not about m2ts. Same result happens with Annex B bitstream. https://elecard-streameye.software.informer.com/download/#downloading
Only 'ffprobe -show_frames' says "co located POCs unavailable".
Fplay does too.
I can parse TS (as in MOV files)
Mov files are mp4.
comment:34 by , 4 months ago
Replying to Balling:
This stream is even worse, pdr0, as it has PTS of zero on I frame ...
Huh? The first PTS is 504137396.
... but two first B frames should have smaller PTS, as they are presented before I frame and since they cannot be fully decoded actually are just dropped, and are later used for decoding of another, third, B frame. LOL
I think you're looking at Ticket_11055.m2ts. Look at Ticket_11080.m2ts -- I removed the 2 B-frames that were between I's DTS & PTS.
comment:35 by , 4 months ago
Look at Ticket_11080.m2ts -- I removed the 2 B-frames that were between I's DTS & PTS.
And that is wrong. It was correct before that.
comment:36 by , 4 months ago
Replying to markfilipak:
You have a parser for MPEG2TS? Actual physical bits would help me to see what you're writing about. I write that because it seems to me you (or MPEG) are mixing stream frames, which have only timestamps and not frame numbers, with something that goes by frame numbers. I suspect they're actually frame offsets, not frame numbers. I suspect they're used for transforming slices between reference slices in other frames as opposed to transforming while frames. I suspect this has something to do with POC, which I find to be 'prescribed' rather than 'described', and so, seems rather vague and subject to interpretation.
Stream parser like Stream Eye has been mentioned several times. It was first thing I mentioned in 11055 .
This link is from Iain E. Richardson's , “The H.264 Advanced Video Compression Standard” . It's a good book, and the author is a known expert
PLEASE READ! This explains frame_num and POC in simple terms
https://www.vcodex.com/h264avc-picture-management
In the simplest terms - Frame_num refers to decoding order . POC refers to the display order . Both frame_num and POC are found in the slice headers and refer to the actual h264 video stream, not something container specific like m2ts , ts, mp4, mkv etc... Frame_num has to increase by 1 every frame until the next IDR, POC does not . But if you get either frame_num or POC decreases, the stream does not conform to specifications .
The most common cause of decreases in frame_num or POC types of non conforming streams are.... you guessed it... bad cuts and appends. e.g. An "I" frame is mistaken for IDR
Both frame_num and POC reset to zero at each IDR. Since you have no IDR's you have wild frame_num (243) compared to the encoded frame count (97), because you are missing frames up to the previous IDR from the original stream. ie. your 1st "i" frame is frame 243 compared to it's own IDR (frame zero) in the original stream. The decrease in POC and frame_num are from the appended section, which was closer in proximity to it's own IDR (fewer missing frames, thus a lower value) . Thus, you must cut on IDR boundaries as explained earlier , or re-encode the affected section before joining if you needed frame accurate cuts. The re-encoded section places a new IDR, the frame_num and POC are reset to zero, and that section is a valid video sequence. (It's a bit more complicated than that, you generally have to encode using similar settings for a proper join, and if it was for authored BD, there are potential VBV issues)
2 - I have a proposal: Since this is going beyond a bug report and the time you are putting in is growing, how about I pay you? Money for instruction. That would only be fair to you and I'm willing to do it. If such instruction does reveal an actual bug, wonderful. If it doesn't that's fine too.
You want a teacher who intimately knows the specs. Sorry, that's not me.
Bottom line - Ticket_11055.m2ts, Ticket_11080.m2ts are invalid streams for the reasons previously posted. If you post a valid spec video sequence, and there are ffmpeg timestamp issues, then it should be investigated.
follow-up: 38 comment:37 by , 4 months ago
pdr0, can you show somehow with some analyser that the two B frames that he removed were actually needed to decode that frame after non-IDR I frames (4th frame needs 1st and 2nd frame from sample 11055)?
As I understand this is some complex relation that is not obvious, Elecard does not print that any frames depend on those two first B frames, but clearly they must as shown by mpv. (mpv click . 1 time, that second frame (4th actual frame) is shown corrupted.)
comment:38 by , 4 months ago
Replying to Balling:
As I understand this is some complex relation that is not obvious, Elecard does not print that any frames depend on those two first B frames, but clearly they must as shown by mpv. (mpv click . 1 time, that second frame (4th actual frame) is shown corrupted.)
On the right side? That's not corruption. That's fade-out on a burning torch. Would you like a longer clip so you can get a better sense of the material around the splice?
Balling, I'm having difficulty. Would you kindly identify the frame you call "second frame".
Below is Ticket_11055.m2ts:
504129888 504133642 504137396 504141150 504144903 504148657
/ / / / / /
PTS order B B I B B P..
___________________________/ ___________________________/
/ /
DTS order I B B P B B P..
\ \ \ \ \ \ \
504126135 504129888 504133642 504137396 504141150 504144903 504148657
Below is Ticket_11080.m2ts:
504137396 504141150 504144903 504148657
/ / / /
PTS order I B B P..
_______/ ___________________________/
/ /
DTS order I P B B P..
\ \ \ \ \
504133642 504137396 504141150 504144903 504148657
You see? I removed the open Bs and rewrote the I-frame's DTS. I cannot discern IDR-frames v non-IDR-frames, and apparently neither can FFmpeg or FFprobe.
PS: Now I see your corruption. Ticket_11080.m2ts's B-frames at PTS 504141150 & 504144903 look like they have some funky slices.
follow-up: 41 comment:39 by , 4 months ago
I cannot discern IDR-frames v non-IDR-frames, and apparently neither can FFmpeg or FFprobe.
You can actually, why is it so hard for you?? ffmpeg.exe -an -i Ticket_11080.m2ts -c copy -bsf:v trace_headers -f null -
then you will see that
[trace_headers @ 000001e4ae7eef40] Recovery Point
and then
[trace_headers @ 000001e4ae7eef40] Slice Header
[trace_headers @ 000001e4ae7eef40] 0 forbidden_zero_bit 0 = 0
[trace_headers @ 000001e4ae7eef40] 1 nal_ref_idc 10 = 2
[trace_headers @ 000001e4ae7eef40] 3 nal_unit_type 00001 = 1
nal_unit_type 00001 = 1
is non-IDR I frame.
comment:40 by , 4 months ago
I'd like to make another comment. It's about the documentation.
showinfo is in https://ffmpeg.org/ffmpeg-filters.html#showinfo. It says
"pts
The Presentation TimeStamp of the input frame"
That is clearly untrue. Clearly, it's the decoder's output. But it's that misleading statement in the documentation that led me to conclude that FFmpeg has a bug.
show_frames is in https://ffmpeg.org/ffprobe-all.html, but it's essentially undocumented.
Oh, I'd also like to make a comment about "non-monotonic DTS". Today I again attempted to find a way to concat my 4 segments. I added "-muxdelay 0" when I saw that having just "-copyts" didn't preserve timestamps. When I added "-muxdelay 0" I got many hundreds "non-monotonic DTS" notices. That does not seem reasonable.
comment:41 by , 4 months ago
Replying to Balling:
I cannot discern IDR-frames v non-IDR-frames, and apparently neither can FFmpeg or FFprobe.
You can actually, why is it so hard for you??
You're joking.
I went to a friend and said, "I need to get to Paris, quick." He said, "I've got just what you need," and led me to his garage. There, spread out on the garage floor were all the parts of a car. "There!" he exclaimed, "Now you have everything you need to get to Paris."
follow-up: 43 comment:42 by , 4 months ago
On the right side? That's not corruption. That's fade-out on a burning torch.
No, it is corruption on top of burning torch. Why cannot you see that?? https://imgur.com/t6mVQ3A
comment:43 by , 4 months ago
Replying to Balling:
On the right side? That's not corruption. That's fade-out on a burning torch.
No, it is corruption on top of burning torch. Why cannot you see that?? https://imgur.com/t6mVQ3A
Yes, you missed my postscript:
PS: Now I see your corruption. Ticket_11080.m2ts's B-frames at PTS 504141150 & 504144903 look like they have some funky slices.
follow-up: 45 comment:44 by , 4 months ago
How did you check your files again if you did not notice such an issue immediately? Did you do framediff between the 11055 and 11088 in YUView? Did you check framemd5? Also, framemd5 shows that the md5 of all the start frames are very different. And after that many frames are actually misordered, though md5 is the same. Which says that first non-IDR frame is actually not a recovery point with exact_match 1!
comment:45 by , 4 months ago
Replying to Balling:
How did you check your files again if you did not notice such an issue immediatelly?
I don't understand. If I didn't notice such an issue, why would I check my files again, eh? I see MPV make lots of mistakes, especially at the top of screens in panning shots.
Do you mean, why did I go from Ticket_11055.m2ts to Ticket_11080.m2ts? Is that what you're asking?
Did you do framediff between the 11055 and 11088 in YUView? Did you check framemd5? Also, framemd5 shows that the md5 of all the start frames are very different. And after that many frames are actually misordered, though md5 is the same. Which says that first non-IDR frame is actually not a recovery point with exact_match 1!
You know, the documentation is the car parts spread out on the floor. There's no tool section. I have no idea what tools there are aside from the three that I use.
I have not the slightest idea what you are 'talking' about. Sorry. I've designed 6U-VME computers (hardware) that are in NATO tanks and aircraft. I've help write industry specs for busses. I've architected supercomputer cache controllers including inventing a then-new cache protocol. I've written drivers and test software for microcontrollers in assembly language. But FFmpeg documentation is beyond my keen.
follow-up: 47 comment:46 by , 4 months ago
There's no tool section.
Yes, I understand. Yuview is here, https://github.com/IENT/YUView/releases/tag/v2.14 unpack it, you need to also download ffmpeg libs and load them in Yuview, unpack this https://github.com/BtbN/FFmpeg-Builds/releases/download/latest/ffmpeg-master-latest-win64-gpl-shared.zip
And load the libraries in this file in settings in Yuview. Then load both two files in Yuview and select them both, selext difference sequence. Now click on difference sequence and here you go. Click second frame, see, it differs.
comment:48 by , 4 months ago
Found this cool analyzer: https://vicuesoft.com/vq-analyzer/userguide/ (see Active/DPB Refs section)
comment:49 by , 4 months ago
Damn, VQ analyzer is a monster. IT CAN FULLY EXPORT ANY FRAME IN ITS ENTERITY! I frame e.g. is 15 MB (!!!) when you convert its binary bitstream into actual names of parameters. This is no NATO tank, this is actual complexity.
Here are all the warnings on your 11055 file (had to convert that to Annex B .h264 file, cause there is a bug apparently, lol, in VQ analyzer version 7.4.0.80497 that it cannon open m2ts files).
https://i.imgur.com/ZhtJsc8.png
(in particular it warns that first nal is actually just non-IDR I frame, while it must be IDR I frame, and it also sees that frame_num has a gap in 176th NAL (non-IDR I frame too) and has a gap in 182 NAL, so I believe maybe you copied the second Open GOP incorrectly).
Also, I can see in hierarchy tab that it has some insane stuff. Basically on this image 2 red dots are the I frames (still non-IDR, remember), blue dots in one line are the P frames, remember the I frames and then P frames can be decoded independently without yet decoding B frames, so it is very nice property. https://i.imgur.com/jwFuQTE.png BUT THEN you can some horrible B frames dependency. You have First level B frames and then 2nd level. And 1st level B frames are kinda normal, but then you have B frame 15/50 depending on 14/13. So 50th actual frame depends on 13th in presentation order. And, yes, they are 15th and 14th in DTS, but still.
Also here are the warnings for your 11088: https://i.imgur.com/pGbfjYE.png (this shows you introduce another frame_num error on the analyzer (cause there is now no frame_num 228 as opposed to 11055), I believe that is another key issue here, clearly you cannot just cut frames with ffmpeg, because this is a warning about gap INSIDE the link between I and P in the initial I, b, B, P, where b is first level hierarchy and B is second level hierarchy in presentation order)
and here is the hierarchy: https://i.imgur.com/LQGR7Jh.png
I think I proved to you that the leading two B frames must be left in the file, why? Because recovery point behind the I frame says that you can decode all frames after recovery point, but you remove the 228 B frame which is as I understand must be decoded, and to decode that you need to decode another B frame.
follow-up: 51 comment:50 by , 4 months ago
Can you give me a file that had more frames before the start of 11055? I wanna see what happens there. Something like another 100 frames should suffice.
comment:51 by , 4 months ago
Replying to Balling:
Can you give me a file that had more frames before the start of 11055? I wanna see what happens there. Something like another 100 frames should suffice.
https://streams.videolan.org/ffmpeg/incoming/11080/Ticket_11055%2C_cut-5sec..cut.m2ts
follow-up: 53 comment:52 by , 4 months ago
Okay, first of all I immediatelly found a bug in Elecard in frame 80 on this sample. Does not happen in ffmpeg or in VQ Analyzer.
Anyway, yes. The first B frame in the GOP (again, in coded order GOP begins with I frame, in presentation/display it is I frame) depends on P frame from the previous GOP, https://i.imgur.com/Y9tduBw.png
Then I verified whether the non-IDR I frame is actually a recovery point, it is. exaqct_match 1 checks out.
comment:53 by , 4 months ago
Replying to Balling:
Okay, first of all I immediatelly found a bug in Elecard in frame 80 on this sample. Does not happen in ffmpeg or in VQ Analyzer.
I'm sorry, are you writing to me, or thinking 'out loud'?
You probably mean you found a bug _using_ Elecard. Why would you call it a bug? That video was mastered by Criterion and the first PTS is 1048560. I see that "1048560" as a sign -- almost a trademark -- of whatever encoder the very best studios use.
Anyway, yes. The first B frame in the GOP (again, in coded order ...
Do you mean in decoder (i.e., DTS) order? "Decoder order" is the term the ITU uses. "DTS order" is the term I use because it's precise: The order is ascending DTSes -- I can't help it, I'm a scientist.
... GOP begins with I frame, in presentation/display it is I frame ...
I don't think so. I'm awfully sure that PTS order is B B I, where B B are two (stale) open Bs.
...) depends on P frame from the previous GOP, https://i.imgur.com/Y9tduBw.png
This is the second or third time you've sent me a link to a picture that I don't recognize as anything. There are no x-y axis units and I don't know what the numbers across the top mean.
Then I verified whether the non-IDR I frame is actually a recovery point, it is. exaqct_match 1 checks out.
Sorry, I don't know what that means or signifies.
comment:54 by , 4 months ago
This is for your amusement -- I'm not bitching...
Do you know what TPS means? Like 90000 TPS for MPEG and 1000 TPS for matroska. It's "ticks per second" of course. The word "timebase" has no meaning. "Ticks per second" has intrinsic meaning.
How about TPF? Like 90090/24 TPF or 90000/25 TPF. Ticks per frame.
PTS = N/TB/FPS becomes PTS = F*TPS/FPS = F*TPF : (frames)*(ticks/sec)/(frames/sec) or better yet (frames)*(ticks/frame). If I could get away with it, I would call "PTS", "picture tick stamp".
I taught the codesmiths who worked for me to use labels based on units instead of abstract words. They told me that it did help them come up with better, faster solutions and that it did help them to see bugs.
comment:55 by , 4 months ago
You probably mean you found a bug _using_ Elecard. Why would you call it a bug?
No, it is a bug in Elecard. Here is how that frame looks like: https://i.imgur.com/bSGfsIG.png
follow-up: 57 comment:56 by , 4 months ago
This is the second or third time you've sent me a link to a picture that I don't recognize as anything. There are no x-y axis units and I don't know what the numbers across the top mean.
This is hierarchy of B frames view. Can be even 4 levels. Again, red dots are I frames (well, light blue here, as I selexted it), blue P, green B, second layer is done with B frames, 3rd too with B frames. X unit is PTS, of course, so the (indeed open) GOP begins with two B frames. The two numbers there are DTS/PTS counting from relative numbers not your absolute numbers (and both from 0).
follow-up: 58 comment:57 by , 4 months ago
Replying to Balling:
This is hierarchy of B frames view. Can be even 4 levels. Again, red dots are I frames (well, light blue here, as I selexted it), blue P, green B, second layer is done with B frames, 3rd too with B frames. X unit is PTS, of course, so the (indeed open) GOP begins with two B frames. The two numbers there are DTS/PTS counting from relative numbers not your absolute numbers (and both from 0).
That diagram, https://i.imgur.com/Y9tduBw.png, looks like nonsense to me. If time is flowing left-to-right, then the 2nd numbers must be PTSes. If that's true, then the 1st numbers must be DTSes. If that's true, then the B-frames are being displayed before they're decoded. That seems like nonsense to me.
Here is what Y9tduBw.png shows.
PTS order B B P B B I B P
\__\/ \__\/ \/
/\ \ /\ \ /\
PTS order P B B I B B P B
\ \ \ \ \ \ \ \
69 70 71 72 73 74 75 76
Here is what's actually in Ticket_11055,_cut-5sec..cut.m2ts.
PTS order B B P B B I B P
______/ ______/ ___/
/ / /
DTS order P B B I B B P B P
\ \ \ \ \ \ \ \ \
\ \ \ \ \ \ \ \ 504054813
\ \ \ \ \ \ \ 504051060
\ \ \ \ \ \ 504047306
\ \ \ \ \ 504043552
\ \ \ \ 504039798
\ \ \ 504036045
\ \ 504032291
\ 504028537
504024783
Now, I believe that inside Ticket_11055,_cut-5sec..cut.m2ts there's I-slices and P-slices and B-slices that reference all over the place. I don't know how those work and I'm not showing them in either diagram above. I'm only showing the DTSes & PTSes.
PS: Actually, B-frames do not have DTSes. I'm showing DTSes for them because FFmpeg shows DTSes for B-frames in MPEG2TSes, but they aren't really there.
PPS: I left out the B-frame at DTS 504028537. It's now corrected. Sorry.
comment:58 by , 4 months ago
B-frames 504039798 & 504043552 are the open Bs. They are part of this GOP:
PTS ..504043552
The later GOP is this:
PTS 504047306..
I may be wrong about that, but I don't think so. H.264 states that the first frame of a GOP shall be an I-frame. ...'nuf said.
follow-up: 60 comment:59 by , 4 months ago
then the B-frames are being displayed before they're decoded
They are only partially decoded, as P frame from previous GOP is absent, Open GOP two first B frames are decoded and dropped.
H.264 states that the first frame of a GOP shall be an I-frame.
H.222 says that. That is old MPEG-2
comment:60 by , 4 months ago
Replying to Balling:
then the B-frames are being displayed before they're decoded
They are only partially decoded, as P frame from previous GOP is absent,
Like I wrote, I don't know how I- P- & B-slices work.
Open GOP two first B frames are decoded and dropped.
How can that be? These are pristine frames that come ahead of my splice. ...May I remind you that this is a video mastered by Criterion?
H.264 states that the first frame of a GOP shall be an I-frame.
H.222 says that. That is old MPEG-2
And H.264 says it, too.
follow-up: 62 comment:61 by , 4 months ago
Check out the 18 seconds picture. There can be 6 levels of B frames depending on the other frames. See, e.g. this crazy sample: https://avc.hhi.fraunhofer.de/mantis/file_download.php?file_id=166&type=bug
The moment we get even some B frames we need to reorder... As first layer of B frames can only depend on I frames and P frames they always must be decoded before. But in presentation they can be after, or before.
comment:62 by , 4 months ago
Replying to Balling:
Check out the 18 seconds picture.
The relationship chart at 0:21 shows 8 levels. Is there something to see there?
See, e.g. this crazy sample: https://avc.hhi.fraunhofer.de/mantis/file_download.php?file_id=166&type=bug
On my computer, that downloads VL_Starcraft_540p60_QP30.bin. I don't know what the 'bin' is. I didn't download it.
The moment we get even some B frames we need to reorder... As first layer of B frames can only depend on I frames and P frames they always must be decoded before. But in presentation they can be after, or before.
"after, or before" what? "after, or before" they're decoded?
comment:63 by , 4 months ago
MPEG's approach is rather naive, wouldn't you say? Each generation is trying to simulate reality but to fit that simulation into (the existing) frame architecture. It's a backwards approach. The next step is to create I-MBs and P-MBs and B-MBs in an independent time domain and eliminate the entire concept of frames. After that, to send just pixel motion transforms. It reminds me of the techniques for modeling telltails on an airplane wing.
FFmpeg would be smart to create a 12000x6000 display with pixels that automatically refresh at around 10 or 12 refreshes per second right now, and to then wrap that in a adaption layer that, on the outside, acts like H.262 or H.264 or H.265 or H.266 etc.
But what do I know? I'm just a simple scientist. :-)
follow-up: 65 comment:64 by , 4 months ago
That file in .h264. Annex B bitsream. Ffplay .bin file and you will see.
"after, or before" what? "after, or before" they're decoded?
I said in temporal order, can be before and after. In closed GOP Unidirectional B frames can still allow for each GOP to have two B frames to start the GOP.
comment:65 by , 4 months ago
Replying to Balling:
That file in .h264. Annex B bitsream. Ffplay .bin file and you will see.
"after, or before" what? "after, or before" they're decoded?
I said in temporal order, can be before and after. In closed GOP Unidirectional B frames can still allow for each GOP to have two B frames to start the GOP.
Okay. I played it. It's a video game. I never ran ffmplay before. What do you want me to see?
follow-up: 67 comment:66 by , 4 months ago
I wanted you to ffprobe it and this this, it has 5 layers of B frames: https://i.imgur.com/HNWXrjb.png
comment:67 by , 4 months ago
Replying to Balling:
I wanted you to ffprobe it and this this, it has 5 layers of B frames: https://i.imgur.com/HNWXrjb.png
Ummm... You want me to ffprobe it? Okay.
C:\Windows\System32>ffprobe g:\VL_Starcraft_540p60_QP30.bin
ffprobe version 2024-05-20-git-127ded5078-full_build-www.gyan.dev Copyright (c) 2007-2024 the FFmpeg developers
built with gcc 13.2.0 (Rev5, Built by MSYS2 project)
configuration: --enable-gpl --enable-version3 --enable-static --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-li
bxml2 --enable-gmp --enable-bzlib --enable-lzma --enable-libsnappy --enable-zlib --enable-librist --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --en
able-libbluray --enable-libcaca --enable-sdl2 --enable-libaribb24 --enable-libaribcaption --enable-libdav1d --enable-libdavs2 --enable-libuavs3d --enable-libxevd --enab
le-libzvbi --enable-librav1e --enable-libsvtav1 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxeve --enable-libxvid --enable-libaom -
-enable-libjxl --enable-libopenjpeg --enable-libvpx --enable-mediafoundation --enable-libass --enable-frei0r --enable-libfreetype --enable-libfribidi --enable-libharfbu
zz --enable-liblensfun --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-dxva2 --enable-d3d11va --enable-d3d
12va --enable-ffnvcodec --enable-libvpl --enable-nvdec --enable-nvenc --enable-vaapi --enable-libshaderc --enable-vulkan --enable-libplacebo --enable-opencl --enable-li
bcdio --enable-libgme --enable-libmodplug --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libshine --enable-libtheora --enable-libtwolame --
enable-libvo-amrwbenc --enable-libcodec2 --enable-libilbc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-lads
pa --enable-libbs2b --enable-libflite --enable-libmysofa --enable-librubberband --enable-libsoxr --enable-chromaprint
libavutil 59. 19.100 / 59. 19.100
libavcodec 61. 5.104 / 61. 5.104
libavformat 61. 3.103 / 61. 3.103
libavdevice 61. 2.100 / 61. 2.100
libavfilter 10. 2.102 / 10. 2.102
libswscale 8. 2.100 / 8. 2.100
libswresample 5. 2.100 / 5. 2.100
libpostproc 58. 2.100 / 58. 2.100
[h264 @ 0000000000471600] Increasing reorder buffer to 2
[h264 @ 0000000000471600] Increasing reorder buffer to 3
Input #0, h264, from 'g:\VL_Starcraft_540p60_QP30.bin':
Duration: N/A, bitrate: N/A
Stream #0:0: Video: h264 (High), yuv420p(progressive), 960x540, 25 fps, 1200k tbr, 1200k tbn
comment:68 by , 4 months ago
Balling, my friend, if you're hoping I can help you, I'll tell this:
H.264 architecture does not have I- P- or B-frames. If x264 does, then x264 is in error.
H.264 architecture does have I- SI- P- SP- and B-slices.
I think that access units are virtual: network abstraction units (NAL units) that are collections of I- SI- P- SP- and B-slices that are delivered in packets that used to be called "PES".
I think that elemental streams (ES) have been demoted to simply be access unit delivery vehicles.
I think that NAL unit 5 is the physical tag for IDR.
comment:69 by , 4 months ago
I think the lines and arrows are just the references for the AU slices. If an access unit is NAL 5, then it's slices do not reference any slices in AUs that came before in decoder order.
I think PESes no longer exist in H.264.
comment:70 by , 4 months ago
I think that to support cutting on IDRs so that splicing can happen, FFmpeg needs to make new IDRs. It needs to take the AU references: I- SI- P- SP- B-slices that are going to be cut off and make a complete AU out of them and then point the new IDR that's created to those new slices. That could happen without transcoding.
comment:71 by , 4 months ago
Oh, and group of pictures -- what we have been discussing -- no longer exists.
comment:73 by , 4 months ago
Replying to Balling:
Ffprobe -show_frames of course
I don't get it. You want me to run show_frames on a video. On the 5s video I posted? Be my guest. On some other video? On the whole of the 1st episode (about 1:30:00)? What?
I want to help, but you have to be explicit. What do you want?
comment:74 by , 4 months ago
The problem with tools that beautify, like DVB Inspector, is that you don't see everything. I'm starting with a hex editor and DVB Inspector. I am getting some surprises. I'll post a report here when I'm done. I know it is off-topic. Sorry. I don't have any other way to get it to you.
comment:75 by , 4 months ago
DVB inspector is for M2TS. Not for Annex B of H.264 stream itself. There are different, very different things.
And anyway, the issue here is that you dropped frames that have frame_num between the frames you preserved. That is the issue. Nothing can be done about it.
comment:76 by , 4 months ago
| Cc: | added |
|---|---|
| Component: | undetermined → documentation |
| Type: | defect → task |
͏ [ Quote markfilipak @ CE 2024-07-07 15:45:49 UTC:
https://trac.ffmpeg.org/ticket/11080#comment:63
͏ MPEG's approach is rather naive, wouldn't you say?
͏ Each generation is trying to simulate reality but to fit that simulation into (the existing) frame architecture. It's a backwards approach.
͏ The next step is to create I-MB (MacroBlock) and P-MB and B-MB in an independent time domain and eliminate the entire concept of frames.
͏ After that, to send just pixel motion transforms. ]
<^> Video is essentially a sequence of motion pictures.
͏ The frame concept shall regardless exist.
͏ And what you describe are essentially no different:
͏ Only superficially unlike.
͏ See also: https://github.com/mpv-player/mpv/issues/7214#issuecomment-2221924958
͏ [ Quote markfilipak @ CE 2024-07-05 01:39:23 UTC:
https://trac.ffmpeg.org/ticket/11080#comment:41
͏ I went to a friend and said: "I need to get to Paris, quick."
͏ He said: "I've got just what you need."; and led me to his garage.
͏ There, spread out on the garage floor were all the parts of a car:
͏ "There!" he exclaimed, "Now you have everything you need to get to Paris." ]
<^> Not car parts. But airplane most have no idea how to drive.
͏ Prominently this airplane has poorly composed user manual.
͏ And no guarantee shall it decompose apart right mid-air...
͏ I also wonder how long does Imgur keep alike attachment images?
͏ Does that posted 10 years ago still work?
follow-up: 80 comment:77 by , 4 months ago
First of all I agree that, indeed, 11055 sample is more correct with first 4 frames, Open GOP must start with BBI order. (Well, I BB in coding order.) FFmpeg should have no problem (so it is a bug in ffmpeg), IMHO, with decoding 11088 file (even if omit those two initial PTS B frames). Analyzers decode 11055 and 11088 the same and just fine. Still, the consensus is to preserve all frame_nums like in 11055. Or use mp4 container edit lists?
Second of all, stop sending all that stuff to ffmpeg-user list, they do not know what they are talking about except Paul and Michael. This file has no mixed slices. This IS A BLU-RAY, the spec per H.264 on Blu-ray requires 4 slices on each frame, this was a big thing when it was finally added to x264 encoder, which is btw still considered state of the art...
Finally, I think I found the real issue here: see analyser, why is this P frame recogized somewhere in the second cut of frames??? BOTH 2 analyzers behave like this:
Let's open a new bug about how it cannot decode BBI if you cut two first B frames, maybe because of that frame!! https://i.imgur.com/DcWUnnz.png This frame PTS 13 (and logically it should be there) yet it places it somewhere in the end.
comment:78 by , 4 months ago
͏ Though mostly have no idea what you are talking about.
͏ I believe the parsing for alike situations (broken frames) shouldn't outright skip frames:
͏ Best effort should be made to present whatsoever presentable.
comment:79 by , 4 months ago
First two B frames in PTS order are depening on frames from previous GOP. They cannot be fully decoded. You can still see them with
-flags2 +showall
comment:80 by , 4 months ago
First, does FFmpeg have dedicated testers?
Second, I'll reply to what I assume has been directed to me.
Replying to Balling:
Second of all, stop sending all that stuff to ffmpeg-user list, they do not know what they are talking about except Paul and Michael.
No worry... I hoped that Gyan would join in, too. If others contribute foo, I can correct them.
This file has no mixed slices.
Mind you, I know about I- SI- P- SP- and B-slices. By "no mixed slices", do you mean all frames have only I-slices and SI-slices? If so, how do you know that? What tool?
This IS A BLU-RAY, the spec per H.264 on Blu-ray ...
The pattern: /blu-ray/ig, doesn't appear in H.264. Do you mean H.264, Annex B?
... requires 4 slices on each frame ...
By "4 slices" do you mean reference slices or reconstructed slices? How do you know what you say? What tool?
Finally, I think I found the real issue here: see analyser, why is this P frame recogized somewhere in the second cut of frames??? BOTH 2 analyzers behave like this:
Let's open a new bug about how it cannot decode BBI if you cut two first B frames, maybe because of that frame!! https://i.imgur.com/DcWUnnz.png This frame PTS 13 (and logically it should be there) yet it places it somewhere in the end.
I respectfully disagree with your reckoning that the open Bs (i.e. in DTS: I B B) are, all three, part of a single GOP. I contend that the open Bs are part of the previous GOP that was cut off. I also respectfully disagree that I- & B-frames and GOPs exist in AVC -- they do not appear in the text, not at all. That makes 'talking' about them impossible.
My concern is NAL AU-1 (aka "non-IDR" frame) and NAL AU-5 (aka "IDR" frame). I- P- & B-frames are a fiction. GOPs are fiction in AVC.
About tools: I use 5 tools: H.222, H.264, a hex editor, Eric Berendsen's DVBinspector, and Peter Daniel's MPEG-2 TS Packet Analyser [sic]. I rely mostly on the first three. I've found the beautifications that tools make hide the truth and also hide their bugs. DVBinspector is problematical. A hex editor never lies. Here is an example of my parsing:
======================================| TS PACKET
0..3 TS header | 47 HH HH HH
sync_byte | 47 == == ==
| == HH HH H=
| .——' '—————————————.
transport_error_indicator | b--- ---- ---- ---- ----
payload_unit_start_indicator | -b-- ---- ---- ---- ----
transport_priority | --b- ---- ---- ---- ---- //0,none..1,for this PID, this packet has priority over others with '0' transport_priority
PID (Packet Identifier) | ---b bbbb bbbb bbbb ---- //0,PAT..1,CAT..2,TS description table..3,IPMP..4..15,(reserved)..16..8190,network_PID Program_map_PID elementary_PID or other..8191,packet is null -- may also convey PCR
transport_scrambling_control | ---- ---- ---- ---- bb-- //0,payload is not scrambled..1..3,(user-defined)
adaptation_field_control | ---- ---- ---- ---- --bb //0,(reserved)..1,has payload..2,has adaptation..3,has adaptation+payload
continuity_counter | == == == =H //0..15
--------------------------------------| TS_PAYLOAD_____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________
4..187 | //PS..PES..stuffing bytes..private data
And one last thing: Can you help me find where NAL units live? I can't find them in the H.264 syntax.
comment:81 by , 4 months ago
Balling,
I'm now working with 00305.m2ts -- no splices -- and have been for three days. I'm finding strangeness. I'm getting somewhat sidetracked in my search for the 'best' example, sidetracked because every time I change the smallest thing, like duration, everything seems to change. I'm preparing a report that might help you in your quest.
Should I post it here?
comment:82 by , 4 months ago
I contend that the open Bs are part of the previous GOP that was cut off.
No. You are wrong. One of B you cut has bigger frame_num so it is part of Open GOP. Open GOP begins with two B frames, or with I frame and then two preceding in pts time B frames that depend on this I frames and frames in previous GOP. I mean, I provided 10 sources here. This is not a question.
By "no mixed slices", do you mean all frames have only I-slices and SI-slices? If so, how do you know that? What tool?
VQ analyzer
comment:83 by , 4 months ago
͏ Would ͏"-flags2 +showall" affect FFprobe?
͏ For reference on "SI" "SP" "SB" whatsoever...
͏ https://www.researchgate.net/figure/SP-SI-Slices-29-a-SP-slice-is-coded-such-that-efficient-switching-between-different_fig11_322523982
follow-up: 90 comment:84 by , 4 months ago
Yes, it does affect it. See first frame:
ffprobe -flags2 +showall -show_frames -select_streams v:0 -i Ticket_11055.m2ts
comment:85 by , 4 months ago
͏ How much difference comparing to:
͏ https://trac.ffmpeg.org/ticket/11055#comment:72
͏ ?
͏ "showall" says: "Show all frames before the first keyframe."
͏ What about the intermediate frames missing?
͏ Is the default frame-skipping behavior really desirable for FF-*?
͏ (typically only sensible for certain playback)
comment:86 by , 4 months ago
͏ "I- P- & B-frames are a fiction. GOPs are fiction in AVC."
<^> The whole FF-* thing might be a fiction: Only your will at reality.
͏ "FFmpeg timestamps do not consistently agree with packet timestamps"
<^> Partly originated from the misconception mentioned in:
͏ https://trac.ffmpeg.org/ticket/11055#comment:199
͏ https://trac.ffmpeg.org/ticket/11055#comment:172
comment:87 by , 4 months ago
For 00305.m2ts, frame 0,
ffprobe -show_frames says this:
frames.frame.0.pts=1048560
frames.frame.0.pkt_dts=1059821 <==
DVBinspector says this:
pts: 0xFFFF0 (1048560)
dts: 0xFF146 (1044806) <==
Hope this helps -- Mark.
comment:88 by , 4 months ago
I should make this clearly stated. Until 3 days ago, I thought that AVC had I P & B frames. Until 3 days ago, I thought that AVC had GOP with GOP headers and all that. FFmpeg tools sure don't indicate otherwise. My eyes are open now.
comment:89 by , 4 months ago
Balling,
Could you use a 40 second clip of the beginning of 00305.m2ts? I compared it to the original disc file. It is identical except for its last frame.
comment:90 by , 4 months ago
Replying to Balling:
Yes, it does affect it. See first frame:
ffprobe -flags2 +showall -show_frames -select_streams v:0 -i Ticket_11055.m2ts
Thank you, Balling.
Are there any more options that I need to use to keep FF* from hiding stuff from me or silently changing things?
Gyan Doshi clued me about '-copyts' and '-muxdelay 0'. Now you have clued me about '-flags2 +showall'. Are there any others that you know about?
Knowing what's actually in videos is really important to me.
Thanks -- Mark.
comment:91 by , 4 months ago
FFprobe is reporting that all the DTSes are coming 3 frames behind the PTSes. That of course is nonsense. If that's not the source of the flood of "nonmonotonic DTS" notices I sometimes get, I'll eat my shirt.
comment:92 by , 3 months ago
https://github.com/EricBerendsen/dvbinspector/releases/tag/release_1_19_1
Is published. It allows to see TP_extra_header, and its time stamp. It is wrong in 11088.m2ts. Timestamp is not increasing packet after packet, as it should. LOL!
I will say that 1) there were problems like this before with 50 issues in https://github.com/google/ExoPlayer/issues/4951
2) there is such an issue in https://trac.ffmpeg.org/ticket/3705#comment:16 but that even does not have Recovery Point SEI.
3) there is an issue that should be fixed first in ffprobe: #8820 where even reencoding does not help, it breaks the decoder completly, yet playback works, also see sample in #3386 that also shows an issue in ffprobe
4) we are not as perfect in following standards as e.g. -vcodec h264_cuvid as an example #7374