Offloading heavy processing in GetNextAudioBlock()

Audio Plugins
1

Hi, I’m working on an audio plugin that uses pitch shift/time stretching libraries to process blocks of an AudioSampleBuffer during DAW playback.

First, I fill an input buffer by reading sections from various audio files, and set the size for the output buffer:

inBuffer->setSize(2, bufferSize, true, true);
reader->read(inBuffer.get(), inStart, inSamples, inPos, true, true);

outBuffer->setSize(2, (bufferSize * stretchVal));
outBuffer->clear();

Then in GetNextAudioBlock(), I stretch a block of input buffer samples which is written to the output buffer, and then copied into the bufferToFill.

int writePos = 0;
const auto numInputSamples = inBuffer->getNumSamples();
while (inputSample <= numInputSamples - samplesInBlock && writePos < samplesInBlock)
{
    int outSamples = jmin(1024, samplesInBlock - writePos);
    int inSamples = stretcher->GetFramesNeeded(outSamples);
    Array<const float*> inputs; Array<float*> outputs;
    
    for (int channel = 0; channel < inBuffer->getNumChannels(); channel++)
    {
        inputs.add(inBuffer->getReadPointer(channel, inputSample));
        outputs.add(outBuffer->getWritePointer(channel, startSample + writePos));
    }
    
    if (inSamples > -1)
        stretcher->ProcessData((float**)inputs.data(), inSamples, (float**)outputs.data());
   
    inputSample += inSamples;
    writePos += outSamples;
}

for (int channel = 0; channel < outBuffer->getNumChannels(); channel++)
    bufferToFill.buffer->copyFrom(channel, 0, *outBuffer.get(), channel, startSample, samplesInBlock);

startSample += samplesInBlock;

This works with a few audio sources (I can hear correct pitch shift/time stretching happening in real time during DAW playback). However, if I have more than 4 audio sources then it seems like the processing that the stretching library does for each instance forces the audio thread to wait and I hear glitches and crackling during playback.

Is there a more efficient way to offload this kind of real-time processing on multiple buffers? Unfortunately there are very few examples on using stretching libraries in JUCE (and none for real time processing). I had been looking into using some sort of background thread (TimeSliceThread, ThreadPool) but am not sure if this is the right approach for a plugin.

Thanks in advance!!

2

First thing is, are you resizing your buffers in GetNextAudioBlock()?

In general, just moving calculations to a background thread will, if anything, make the problem worse because now the audio thread has to synchronize with the background thread, and the background thread will have lower priority than the audio thread.

I don’t have a great answer for optimizing this computation beyond reading the library’s docs more thoroughly to check for mistakes in your usage, switching libraries, or implementing it yourself.

3

Thanks for the response!

The first code example that I posted where I set the buffer sizes and fill the input buffer is done in prepareToPlay().

Thanks for clarifying that a background thread is likely not the right approach! I am happy not to get into multi-threading!! I am using ElastiquePro for stretching and unfortunately I have not come across any information in the docs or examples online for real-time processing. I’ll read through again though in case I am missing something.

1 Like
4

Your use of Array during the loop is constantly (de)allocating heap memory on the audio thread.

Either change inputs and outputs to be member variables and allocate their storage during your prepare function, or use std::array.

1 Like