AAC to PCM produced a lot of noise!

[kaienfr]

In git-master ffmpeg, converting a AAC audio to PCM produced a lot of noise, even we did resampling from AV_SAMPLE_FMT_FLTP to AV_SAMPLE_FMT_S16.

While in old ffmpeg, aac is converted directly into 16-bit PCM and without any noise. Is it a bug or something? I'd like to know how to produce a proper PCM in the new version of ffmpeg? Thanks a lot!

Here is the information about my test AAC audio which reads by new ffmpeg:

Stream #0:0: Audio: aac, 44100 Hz, stereo, fltp, 122 kb/s

(the format fltp is shown as s16 in old ffmpeg, and flt in decoder)

I've tried three ways to resampling the output FLTP to S16LE,

1. using swr_convert
2. using avresample_convert
3. convert manualy

But all of them yield the same result. The sound quality is really too bad, very slow and out of tune, with a lot of noise too.

My resampling code is as follows, please have a look what's missing

void resampling(AVFrame* frame_, AVCodecContext* pCodecCtx, int64_t want_sample_rate, uint8_t* outbuf){
    SwrContext      *swrCtx_ = 0;
    AVAudioResampleContext *avr = 0;

    // Initializing the sample rate convert. We only really use it to convert float output into int.
    int64_t wanted_channel_layout = AV_CH_LAYOUT_STEREO;

#ifdef AV_SAMPLEING
    avr = avresample_alloc_context();
    av_opt_set_int(avr, "in_channel_layout", frame_->channel_layout, 0);
    av_opt_set_int(avr, "out_channel_layout", wanted_channel_layout, 0);
    av_opt_set_int(avr, "in_sample_rate", frame_->sample_rate, 0);
    av_opt_set_int(avr, "out_sample_rate", 44100, 0);
    av_opt_set_int(avr, "in_sample_fmt", pCodecCtx->sample_fmt, 0); //AV_SAMPLE_FMT_FLTP
    av_opt_set_int(avr, "out_sample_fmt", AV_SAMPLE_FMT_S16, 0);
    av_opt_set_int(avr, "internal_sample_fmt", pCodecCtx->sample_fmt, 0);
    avresample_open(avr);
    avresample_convert(avr, &outbuf, frame_->linesize[0], frame_->nb_samples, frame_->extended_data, frame_->linesize[0], frame_->nb_samples);
    avresample_close(avr);
    return;
#endif

#ifdef USER_SAMPLEING
    if (pCodecCtx->sample_fmt == AV_SAMPLE_FMT_FLTP)
    {
            int nb_samples = frame_->nb_samples;
            int channels = frame_->channels;
            int outputBufferLen = nb_samples & channels * 2;
            auto outputBuffer = (int16_t*)outbuf;

            for (int i = 0; i < nb_samples; i++)
            {
                    for (int c = 0; c < channels; c++)
                    {
                            float* extended_data = (float*)frame_->extended_data[c];
                            float sample = extended_data[i];
                            if (sample < -1.0f) sample = -1.0f;
                            else if (sample > 1.0f) sample = 1.0f;
                            outputBuffer[i * channels + c] = (int16_t)round(sample * 32767.0f);
                    }
            }
            return;
    }
#endif
    swrCtx_ = swr_alloc_set_opts(
            NULL, //swrCtx_,
            wanted_channel_layout,
            AV_SAMPLE_FMT_S16,
            want_sample_rate,
            pCodecCtx->channel_layout,
            pCodecCtx->sample_fmt,
            pCodecCtx->sample_rate,
            0,
            NULL);

    if (!swrCtx_ || swr_init(swrCtx_) < 0) {
            printf("swr_init: Failed to initialize the resampling context");
            return;
    }

    // convert audio to AV_SAMPLE_FMT_S16
    int swrRet = swr_convert(swrCtx_, &outbuf, frame_->nb_samples, (const uint8_t **)frame_->extended_data, frame_->nb_samples);
    if (swrRet < 0) {
            printf("swr_convert: Error while converting %d", swrRet);
            return;
    }
}

I've also noticed that in ffplay, the aac audio is playing without noise. So what is the problem it could be? what is the right way of the conversion? is there something wrong in in decode part or resampling part or something else (such as we must use filter...)? Please help!

Thanks a lot in advance!

[Carl Eugen Hoyos]

Replying to kaienfr:

In git-master ffmpeg, converting a AAC audio to PCM produced a lot of noise, even we did resampling from AV_SAMPLE_FMT_FLTP to AV_SAMPLE_FMT_S16.

You may of course use swresample directly but consider using the aresample filter instead.

While in old ffmpeg, aac is converted directly into 16-bit PCM and without any noise. Is it a bug or something?

No, it is a performance improvement.

swrCtx_ = swr_alloc_set_opts(

NULL, swrCtx_,

Why aren't you calling swr_alloc()?
If you want to use libswresample directly, you can look at libavfilter/af_aresample.c for an example on how to use the library.

I've also noticed that in ffplay, the aac audio is playing without noise.

Did you also test ffmpeg -i file.aac out.wav?
Please understand that this is a bug tracker, not a support forum, see https://ffmpeg.org/contact.html

[kaienfr]

Thanks a lot for your response. I will try call swr_alloc() for testing, but I am not sure if it will work since in fact, it works quite well in the old version for a long time...

I have asked this question on several forums such as stack overflow ​http://stackoverflow.com/questions/22822515/ffmpeg-resampling-from-av-sample-fmt-fltp-to-av-sample-fmt-s16-got-very-bad-so

and chinavideo ​http://bbs.chinavideo.org/forum.php?mod=viewthread&tid=19551#lastpost

but always no response and no correct solution. It's not only me but many people have the same problem when converting from AAC to PCM and only in NEW ffmpeg libs (watched from google), and always no solution. That's why I doubt it could be a bug and ask question here.

I will try it and give you feedback soon. Thanks a lot in advance!

[kaienfr]

Nothing changed with swr_alloc()

Here is the comparaisons of decoded output with and without resampling produced by the same code with different ffmpeg versions. The figure is made with "cool edit pro"

1. old ffmpeg without resampling (44100, stereo, s16le)

2. new ffmpeg without resampling (44100, stereo, 32bit IEEE float 0.24, Impossible to open with s16le since they are all noise, if you want to see, I can also show you that picture...):

3. old ffmpeg with resampling (44100, stereo, s16le):

4. new ffmpeg with resampling (44100, stereo, s16le):

You can observe quite a different between the outputs. Old ffmpeg is all right for both decode and resampling, well new ffmpeg does not work well on neither of them.

I haven't tried filter, I will do it later, and tell you what happened. Thanks again for your futher helps!
(PS: testing on ffmpeg.exe and ffplay.exe are all right.)

[kaienfr]

I've tried your suggestion to use aresample filter, this is the code:

	ReSampleContext *rs_ctx = NULL;
        // resample to 44100, stereo, s16
	rs_ctx = av_audio_resample_init(
		2, pCodecCtx->channels,
		44100, pCodecCtx->sample_rate,
		AV_SAMPLE_FMT_S16, pCodecCtx->sample_fmt,
		16, 10, 0, 1);
        outbuff = (uint8_t*)av_malloc(AVCODEC_MAX_AUDIO_FRAME_SIZE);

       .....................

	// resampling
	after_sampled_len = audio_resample(rs_ctx, (short *)outbuff, (short *)pFrame->extended_data, pFrame->nb_samples);

I've gotten the following errors at the function audio_resample:

[audioresample @ 023AC060] Audio sample format conversion failed

The same code run perfactly with old ffmpeg and got wonderful result！

Well, what kind of problem this should be in new ffmpeg? Obviously, there is something abnormal in the new version.
If the way to use it is changed, PLEASE clarify it. I am looking forward to your help. Thank you very much indeed!

[kaienfr]

Well I figure it out. comparing to the old ffmpeg, It seems that in new ffmpeg, each frame is twice larger than old one, so I guess that the output size could be larger estimated in new version. So I tried devided the size of the resampled data by 2, well I got the perfact audio

This difference seems for aac audio between old and new ffmpeg. Hope it will helpful to people who is still under the trap of the new ffmpeg issues. Thanks all the same cehoyos for your apply to my question.