How to handle latency in pitch shifter (rubberband)

Hi guys,

Im trying to implement a pitch shifter but I’m facing some difficulties regarding the real time processing.

I’m using RubberBand which sound great when I change the pitch scale but I’ve some question about the latency.

Rubberband give 2 functions:

/**
* Return the processing latency of the stretcher.  This is the
* number of audio samples that one would have to discard at the
* start of the output in order to ensure that the resulting audio
* aligned with the input audio at the start.
*/
getLatency();

and

/**
* Ask the stretcher how many audio sample frames should be
* provided as input in order to ensure that some more output
* becomes available.
* 
* If your application has no particular constraint on processing
* block size and you are able to provide any block size as input
* for each cycle, then your normal mode of operation would be to
* loop querying this function; providing that number of samples
* to process(); and reading the output using available() and
* retrieve().  See setMaxProcessSize() for a more suitable
* operating mode for applications that do have external block
* size constraints.
*/
getSamplesRequired();

However to get a processing without have some empty buffer in the middle of the signal from the start of the stream, I need to have a total latency of getLatency() + getSamplesRequired() + processBlockSize

So for my maximum required sample case (lowest pitch scale possible at 0.5), I’ve 2049 + 2048 + 512 latency sample.
But If I look only at the getLatency() or getSamplesRequired() I should have 2049 or 2048.

Note that I use setMaxProcessSize(spec.maximumBlockSize) which is recomanded when w

Here is my code with the prepare function:

void PitchShifter::prepareRubberBand(const juce::dsp::ProcessSpec& spec) noexcept {
  // Used to transfert data between context and rubberband
  rubberBandBuffer_ = 
  juce::AudioBuffer<float>(spec.numChannels, spec.maximumBlockSize);

// Init rubberband with options
  auto stretcherOptions = 0;
  stretcherOptions |= RubberBand::RubberBandStretcher::OptionProcessRealTime;
  rubberBandStretcher_ =
  std::make_unique<RubberBand::RubberBandStretcher>(spec.sampleRate,
                                                    spec.numChannels,
                                                    stretcherOptions);

  rubberBandStretcher_->setPitchOption(RubberBand::RubberBandStretcher::OptionPitchHighConsistency);
  rubberBandStretcher_->setMaxProcessSize(spec.maximumBlockSize);

  // Set the lowest pitch scale in order to get the max (i.e worst) required
  // number of sample.
  rubberBandStretcher_->setPitchScale(0.5);
  const auto stretcherLatency = rubberBandStretcher_->getLatency();
  const auto requiredSamples = rubberBandStretcher_->getSamplesRequired();
  
  // Get back to the default pitch scale.
  rubberBandStretcher_->setPitchScale(1);

  const auto latency = static_cast<int>(stretcherLatency)
  + static_cast<int>(requiredSamples)
  + spec.maximumBlockSize;

  auto silenceBuffer = juce::AudioBuffer<float>(2, spec.maximumBlockSize);
  auto silenceBlock = juce::dsp::AudioBlock<float>(silenceBuffer);
  silenceBlock.fill(0);

  // Fill the buffer here to avoid empty buffer during the process.
  auto processedSampleCount = 0;
  while(processedSampleCount < latency) {
    rubberBandStretcher_->process(silenceBuffer.getArrayOfReadPointers(),
                                  silenceBuffer.getNumSamples(),
                                  false);
    processedSampleCount += silenceBuffer.getNumSamples();
  }
}

and the process function:

void PitchShifter::processRubberBand(const ContextReplacing& context) noexcept {
  auto ioBlock = context.getOutputBlock();
  auto sampleCount = static_cast<int>(inputBlock.getNumSamples());
  auto channelCount = inputBlock.getNumChannels();

  for (int channelIndex = 0; channelIndex < channelCount; channelIndex++) {
    auto source = ioBlock.getChannelPointer(channelIndex);
    rubberBandBuffer_.copyFrom(channelIndex, 0, source, sampleCount);
  }

  const auto required = rubberBandStretcher_->getSamplesRequired();
  const auto availableSampleCount = rubberBandStretcher_->available();

  rubberBandStretcher_->process(rubberBandBuffer_.getArrayOfReadPointers(),
                                sampleCount,
                                false);

  if (availableSampleCount > sampleCount) {
    auto retrievedSamples =
    rubberBandStretcher_->retrieve(rubberBandBuffer_.getArrayOfWritePointers(),
                                   sampleCount);
  } else {
    // If I don't process in the prepareRubberBand function,
    // I go here and get empty buffer in the real time at the begining and
    // 3 times after few processing in the begining then it's ok.
    rubberBandResultBuffer_.clear();
  }

  ioBlock.copyFrom(rubberBandBuffer_);
}

Is it normal that I have to use the getLatency() + getSamplesRequired() + processBlockSize number of sample to have the right process latency?

Thanks