AudioProcessor using RubberBand Stretcher

windows
audio

#1

Hi,

I’m working on a AudioProcessor that manage a RubberBandStretcher instance. I just confused with the RubberBand process/retrieve methods. In the main AudioProcessor processBlock method, I need to push the buffer in the RubberBand process :

void MyProcessor::processBlock(AudioBuffer<float>& buffer, MidiBuffer& midiMessages)
{
    rb->process((const float* const*)buffer.getArrayOfReadPointers(), buffer.getNumSamples(), false);
}

But what about the RB retrieve method ? How to put the stretched result into my buffer while the available output data must have different size than my AudioProcessor buffer ?

Thank you very much for your help.

Friendly,
Max


#2

You will probably(*) need to do additional buffering (queue or ring buffer) yourself. Note that you need to be careful about the output getting too long, if you are actually attempting something like time stretching the input coming into the plugin. The situation might be more under control if you are not time stretching but only pitch shifting with the library. (You still may need to do some buffering of your own.)

(*) I haven’t checked for a while how the RubberBand API works, will it only output buffers of a size it itself determines. If it does that, then you will need to do that buffering of your own.


#3

Hi Xenakios,

Thank you for your answer.

To be more precise, but I’m newbie on the RubberBand API :

  • Make an instance of RubberBandStretcher in the AudioProcessor prepareToPlay(), like that (force Stereo 2 channels) :

  • rb = new RubberBand::RubberBandStretcher(sampleRate, 2, OptionProcessRealTime | OptionStretchPrecise);

  • Set the maximum process size (always in prepareToPlay);

  • rb->setMaxProcessSize(maximumExpectedSamplesPerBlock);

  • If my Time Ratio is 1.0 (no stretch), in the processBlock() that code works and just do nothing (bypass) :

  • rb->process((const float* const*)buffer.getArrayOfReadPointers(), buffer.getNumSamples(), false);

  • rb->retrieve((float *const *)buffer.getArrayOfWritePointers(), buffer.getNumSamples());

  • When I set the Time Ratio to 1.5, slow down is effective but few seconds latter the audio output turn buzzy like when denormals values problem.

I think the problem comes from the use direct raw pointers on the sample buffer array for the in/out RB process.

So, I need a private buffer to manage what exactly ? The output from retrieve ? In all case, each time the processBlock() method is called I need to fill the RB process with my buffer and write the output in the same buffer. So that’s why i’m confused with that kind of process, maybe if someone have a short example code that would be wonderful.

Thank’s a lot.
Max


#4

There’s probably no short example code around because what you are trying to do (time stretching real time input) is pretty complicated and the usual plugin formats like VST or AudioUnits are not really meant for doing that kind of stuff anyway.

I also don’t know the RubberBand API that well, but based on other similar libraries I’ve used, I can guess it’s not going to be straight forward to use. They usually want inputs and deliver outputs of sizes that don’t match the plugin processing buffer size.


#5

Yes, you’re right. I don’t understand why RubberBand API do not manage that kind of extra buffer in is private implementation. Flexibility ? Perhaps, but the API come’s with only 3 examples : command line tool (offline process), LADSPA & Vamp plugins (online process) (but I do not know any of these kind of plugins).


#6

Doesn’t the LADSPA plugin example show how it should be used? IIRC LADSPA is a plugin format similar to VST and such.

edit : Yeah, the LADSPA code is what I expected, it uses a ring buffer etc to deal with the buffer size mismatch issues. You will need to look into that code and adapt it into your JUCE based plugin. (Obviously respecting any licensing involved.)


#7

Maybe it’s me, but a time stretch algorithm set to 1.5 that you send 1 sec. of audio would return 1.5 secs audio. If input and output are realtime signals, you need to invent a time machine… no?


#8

Not if the audio is slowed down, that doesn’t in any way violate the laws physics! But of course some tricky buffering is involved in that case. (And obviously the plugin must at some point decide to not buffer any more of the slowed down output or there’s a risk of running out of memory.)


#9

I have check the code few days before, but I don’t understand to port the code for my AudioProcessor. Checking the code right now, I see the use of a ringbuffer.


#10

Ok, I thought so. In which case you combine the signal coming out with a resampler, so that you end up with the same amount of samples. The result is a pitched signal rather than a time stretched…


#11

No, it’s not about that. It’s entirely possible to slow down real time inputs. :slight_smile: (IMHO it’s not that useful to actually do, I’ve tested it myself, but people regularly come up with that idea.)


#12

The answer is latency no ? When you stretch you audio signal, a latency is involved.


#13

Sorry for nit-picking, but slowing down is not the same like making it longer… there is a relation though.
My point was, the stretcher stretches the signal, resulting in the original problem: more samples, than needed. To remedy that feed these samples into a resampler with the inverted factor, so you end up with a pitched signal with the correct number of samples.


#14

Right, the stretching obviously produces more samples than needed for the current processBlock call, therefore it has to be buffered to be outputted later. (Then appears the problem that the plugin has to stop accepting audio from the stretcher at some point so that all the memory of the computer is not used. I am not considering here the horrible details of how to actually buffer an increasing amount of audio etc. The simplest is to just set some limit, reserve buffer space for that and buffer until that space has been used.)


#15

Right, either to be outputted later or resampled, if a pitch was the actually intended effect, only @maxprod2016 knows :wink:


#16

Pitch shifting doesn’t require any resampling with RubberBand, it’s built into the library.


#17

Ok, I don’t know Rubberband specifics. I was answering in general to time stretchers. I think you guys can take it from here.

Good luck.


#18

So it’s not real-time at all then?


#19

Disregarding latency issues (which it does have), when only pitch shifting with it, it can do real time processing. It is also capable of time stretching well within real time constraints in terms of CPU use but trying to do the time stretching within a plugin is just a very tricky thing to implement. (Unless you preload/capture the whole audio into the plugin before it’s processed.) It’s much better suited for things like time stretching audio clips/items in a DAW. (For example Reaper has added RubberBand as an option for that.)