How do MIDI note event timings relate to audio buffer processing? ie. How do we synchronize the buffer samples to the MIDI event times?

General JUCE discussion
1

I am trying to understand better how MIDI events interleave with block rendering.

Is the whole MPESynthesiser a single threaded class?

Let’s say you have some basically default MIDI behaviors:

void MPESynthesiser::startVoice (MPESynthesiserVoice* voice, MPENote noteToStart)
{
    jassert (voice != nullptr);

    voice->currentlyPlayingNote = noteToStart;
    voice->noteOnTime = lastNoteOnCounter++;
    voice->noteStarted();
}

void MPESynthesiser::stopVoice (MPESynthesiserVoice* voice, MPENote noteToStop, bool allowTailOff)
{
    jassert (voice != nullptr);

    voice->currentlyPlayingNote = noteToStop;
    voice->noteStopped (allowTailOff);
}

//==============================================================================
void MPESynthesiser::noteAdded (MPENote newNote)
{
    const ScopedLock sl (voicesLock);

    if (auto* voice = findFreeVoice (newNote, shouldStealVoices))
        startVoice (voice, newNote);
}

void MPESynthesiser::noteReleased (MPENote finishedNote)
{
    const ScopedLock sl (voicesLock);

    for (auto i = voices.size(); --i >= 0;)
    {
        auto* voice = voices.getUnchecked (i);

        if (voice->isCurrentlyPlayingNote (finishedNote))
            stopVoice (voice, finishedNote, true);
    }
}

Then for rendering your voices you have:

void MPESynthesiser::renderNextSubBlock (AudioBuffer<float>& buffer, int startSample, int numSamples)
{
    const ScopedLock sl (voicesLock);

    for (auto* voice : voices)
    {
        if (voice->isActive())
            voice->renderNextBlock (buffer, startSample, numSamples);
    }
}

And in the voice, something like:

	void renderNextBlock(AudioBuffer <float>& outputBuffer, int startSample, int numSamples) override {

	for (int sample = 0; sample < numSamples; sample++) {
		output = //solve output;

		for (int channel = 0; channel < outputBuffer.getNumChannels(); ++channel) {
			outputBuffer.addSample(channel, startSample, output); ///////MAIN OUTPUT HERE

		}
		//INCREMENT SAMPLE
		++startSample;

}

Let’s say you have a big buffer (for illustration sake) of 2048 at 44100 Hz which would be 46 ms. 46 ms is a long time (intentionally in this case).

I presume MIDI events are coming in on a per sample buffer basis as well? Right? Or how are they?

What I mean is, let’s say someone pressed a note midway through the last 2048 sample buffer which we are only just now rendering. How (if at all) does this system account for this?

How can you know at what stage of the 2048 samples the note was pressed so you can correctly trigger the note behavior at that time point when running your render loop?

Or are most people (wittingly or unwittingly) just processing all their MIDI events at the beginning of each audio buffer and then looping the buffer without any attention to the exact alignment of things?

The only logical way I can imagine it to work properly is if we have:

| 2048 samples of audio to fill |===> RUN RENDER LOOP ON BOTH IN SYNC
| 2048 samples of MIDI data     |===> RUN RENDER LOOP ON BOTH IN SYNC

If we have a “buffer” of same sample size as the audio buffer of our midi data, then we can process it in a synchronized fashion to our audio rendering.

In the case of a long buffer, if someone arpeggiates three notes during the time of that one buffer, we still get an arpeggiation out, rather than them being clumped out as one at the start of the next buffer.

How is it supposed to work? How does it work? Does what I describe make sense?

2

Take a look at processNextBuffer in juce_Synthesiser.cpp

juce::MidiBuffer provides the method findNextSamplePosition which finds the sample position of the next MIDI message greater than or equal to the position passed into the method’s argument and returns a juce::MidiBufferIterator instance at that position

processNextBuffer then renders the buffer’s audio between the start sample and the position of that message, gets the next message’s position, and then repeats until the end of the audio buffer (and then repeats when the buffer loops back through)

Looking through that method in juce_Synthesiser.cpp is a good way to get a general idea of how to sync MIDI messages with the sample position of your audio buffer. I’ve recently had to figure out how to do that as well, since my current project mixes standard C++ code for synthesis/dsp with JUCE for MIDI/audio IO and GUI

2 Likes