Fixing av_seek_frame

[DonMoir]

libavutil 51. 16. 1 / 51. 16. 1
libavcodec 53. 16. 0 / 53. 16. 0
libavformat 53. 12. 0 / 53. 12. 0
libavdevice 53. 4. 0 / 53. 4. 0
libavfilter 2. 43. 2 / 2. 43. 2
libswscale 2. 1. 0 / 2. 1. 0
libpostproc 51. 2. 0 / 51. 2. 0

av_seek_frame fails in several cases. This is an attempt to write a seek function that works as expected. It may be that avformat_seek_file will be an attempt to fix all this, but currently it just calls av_seek_frame since read_seek2 does not exist. So mostly this needs to be a rewrite of libavformat\utils.c.

I see that alot of the failures occur because the AVStream index_entries is NULL or none of the entries are flaged with AVINDEX_KEYFRAME, but there are other reasons for failure and thats going to require more digging.

This discussion is probably going to end up being lengthy. I have several files that fail but they fail for various reasons. If you want I will just put it all on this ticket, or open a new ticket for each case, or we can just communicate by email. Let me know.

This first case is the simplest I have. It has index_entries but none of them are flagged with AVINDEX_KEYFRAME. It has read_seek but that returns -1. It ends up calling seek_frame_generic which calls av_index_search_timestamp and that fails since none of the entries are marked with AVINDEX_KEYFRAME. So the end result of av_seek_frame is -1. You cannot just call av_seek_frame with AVSEEK_FLAG_ANY because that will just put you in the middle somewhere and that will produce incorrect results when you decode it. For looping back to the beginning, using AVSEEK_FLAG_ANY will work since the reality here is the first frame is the key frame. In order to correcly seek into this file at an arbitrary position, I suppose you would need to decode the first frame and then proceed to the timestamp of interest. The attached file, filecopy.avi, can be tested with ffplay by using the left arrow key and you will see it says 'error while seeking'.

This is just the first case. I have several others but maybe by fixing a few these it will cover most cases.

Another note for now is when av_seek_frame_binary is called, there are 2 variables called pos_min and pos_max. I have files where av_seek_frame_binary is called and either 1 or 2 of the variables are never initialized. Then av_gen_search is called with the unintialized variables. I will get back to this later with file examples.

[DonMoir]

A lot of the seek errors I am seeing are with MPEG1 files. Not all of them fail though. I get some failures with MPEG2 as well. I am still looking into this. Most of the other formats I have tried work pretty good but I have this CODEC_ID_H264 case that fails. (AspectRatioHB.mpg). This particular file has 1 keyframe in the first frame and it is marked as such. The timestamp for this frame is negative which might have something to do with the failure but I don't know. Any attempt to seek directly to this first frame will fail. I have tried serveral different ways to seek to the frame but it always ends up with incorrect results. Other ways might be avio_seek and update the timestamp etc. Like the seek may return -1 or if I rearrange things seek returns success but then on decode I might get a gray frame or similiar. The rest of the H264 files I have work fine.

[DonMoir]

On that AspectRatioHB.mpg file, if I do a avio_seek on the postion of the first frame and then update the timestamp (which is negative for that frame), on the first read using av_read_frame, you will not get that first frame packet. You will get some packet in the middle and that will produce incorrect results. It's like it skipped packets and the key first frame was never decoded. Also, if I do not seek on the file and just start reading, then all is good and I get the first packet correcly with negative timestamp etc.

[DonMoir]

With dumbdog.mpg and I think several other MPEG1 files, the index_entries are all flagged with AVINDEX_KEYFRAME. When you start reading, the packets are flagged with AV_PKT_FLAG_KEY. When you start decoding after a seek and you get a finished frame, the AVFrame key_frame field is zero. Can someone explain why this is ? It seems since I have seeked to a keyframe and I start reading and decoding at the seeked position, I should be getting a keyframe. It appears that the index_entries being flagged as AVINDEX_KEYFRAME is a bug and they do not really represent keyframes?? So you end up displaying a gray or incomplete frame.

With ffplay, the seek sort of works when you hit left arrow, but right arrow will cause it to stop playing. The fact the left seek kind of works with ffplay is more accidental then correctly working. I know why that is but I don't think it's relevant to the problem.

[DonMoir]

I am side tracked at the moment having to do some other work but this issue is important to me and I am sure for others. The first pass at this was to identify some of the problems. I will be getting back to this soon and will dig deeper to see if I can come up with some solutions. In the previous posts, there are some comments or questions made and if you have any information on those it would be useful to know what you think.

[Michael Niedermayer]

A patch that improves the seeking with ffplay in dumbdog.mpg is on ffmpeg-dev since a few hours

[Ganesh Ajjanagadde]

filecopy.avi - seeking fixed in 83d6ad3616cac9ecfe23d83822b2e9d10a8c985d.