Weird MIDI output latency on Windows?

General JUCE discussion
1

On Mac my MIDI output latency is fabulous, but when I tried my software on Windows, I get about half a second latency between when I send the MIDI data vs. when it actually comes out of my MIDI interface. I’ve managed to limit the possible culprits of this weird behavior to two options:

  1. Time::getMilisecondTimerHiRes() // Used for setting MIDI data output time
  2. MidiOutput::sendBlockOfMessages()

If I subtract about 500 ms from the output of getMilisecondTimerHiRes(), things start sounding really nice. I want to know from where the extra latency comes from. So is the issue with the Time’s output or the sendBlockOfMessages which adds some extra latency on Windows?

Why don’t these work exactly the same way on Mac and Windows? How to fix this issue?

1 Like
2

Anyone have any idea what could cause this weird behavior?

3

I did some debugging and here’s what I’ve found so far:

MidiOutput::sendBlockOfMessages() takes a millisecond value and samplerate as a parameters to calculate when the MIDI messages should actually happen. MidiMessage’s samplePosition is also used to calculate this information.

I have set all the outgoing MidiMessages’ samplePositions to 0.
The MidiOutput::sendBlockOfMessages() takes the parameter “millisecondCounterToStartAt” which I have set to what comes out of Time::getMillisecondTimerHiRes(). All the MidiMessages in the buffer use the exact same value for millisecondCounterToStartAt.

With this setup I get about 3 second latency to all the MIDI messages I sent out with the above method.

MidiOutput::run() processes these MidiMessages which have been earlier converted into PendingMessages by MidiOutput::sendBlockOfMessage(). That conversion also calculates the eventTime automatically which is used later on in the process.

MidiOutput::run() gets its current time from Time::getMillisecondCounter() to calculate the actual time when the outgoing MIDI data should be sent out. This means that there are two different methods being used to calculate the output time:

I’m using Time::getMillisecondTimerHiRes() to define when the MIDI data should be sent out.
MidiOuput:run() uses Time::getMillisecondCounter() which is a different method.

This seems to somehow result into about 3 second latency between the moment I send MIDI data out with as small latency as possible, to the data actually coming out from my MIDI interface. Are those two millisecond counter about 3 seconds out of sync with each other on Windows platform or is there something else at play here?

Rebooting my computer drops the latency to about 0.5 seconds.
There seems to be a 500 ms “timeToWait” default latency in MidiOutput::run() for some reason. Is my issue somehow related to what is being done with that timeToWait default value?

4

I replaced the MidiOutput::run()'s line:
auto now = Time::getMillisecondCounter();

with the following:
auto now = Time::Time::getMillisecondCounterHiRes();

And the timing becomes immediate and razorsharp. I think I found your bug in JUCE.

Please confirm that this is the proper way to fix the issue. If it is, please add this fix to JUCE.

EDIT:
That fix seems to create a new problem:
When I play MIDI notes really fast one after the other with fixed time intervals, I get one note playing and after a short while a super fast burst of MIDI notes. This repeats again and again as I feed a continuous fast stream of MIDI notes into the MidiOutput.
So my fix won’t work after all but created a new one.

Please advice.

5

Testing with JUCE’s own DemoRunner’s MidiDemo, I get the following behavior:

Pressing notes on MIDI controller the demo shows incoming midi messages. Those same messages won’t get sent to the selected output devices. Only when I press the drawn keyboard on computer screen with my mouse, then I get MIDI notes coming out of my MIDI interface. So there’s something wrong with the JUCE demo also, I believe.

Next I’ll try writing my very own minimal MIDI I/O test and see what happens.

7

AudioAppDemo.zip (14.1 KB)

Attached to this message is a short 200 line MIDI I/O test. I made it as minimal and short as possible. It demonstrates clearly the issue I’m having.

Could someone compile and test the application on their own Windows computer and let me know if they also have the clearly audible (at least half a second) MIDI lag between pressing a note from keyboard to actually it coming back out from their MIDI interface and you hearing the note playing?

Below is the same piece of C++ code:

#pragma once

class AudioAppDemo   : public AudioAppComponent
{
public:
    AudioAppDemo()
    {
        setAudioChannels (0, 2);

        createGUI();
        setSize (800, 600);

        midiInputChanged();
    }

    ~AudioAppDemo() override
    {
        disableAllMidiOutputs();
        disableAllMidiInputs();
        shutdownAudio();
    }


    void disableAllMidiInputs()
    {
        auto midiInputList = juce::MidiInput::getAvailableDevices();

        // Ensure all MIDI inputs are disabled
        for (auto& x : midiInputList)
            if (deviceManager.isMidiInputDeviceEnabled(x.identifier))
                deviceManager.setMidiInputDeviceEnabled(x.identifier, false);

        // Remove previously existing callback
        deviceManager.removeMidiInputDeviceCallback("", &midiInputCollector);
    }


    void midiInputChanged()
    {
        disableAllMidiInputs();

        int itemId = midiInputSelection.getSelectedId();
        jassert(itemId != 0);        // Ensure our test has a valid comboBox item selected
        int listIndex = itemId - 1;

        auto midiInputList = juce::MidiInput::getAvailableDevices();

        // Enable the requested MIDI input and get data from all enabled MIDI inputs
        deviceManager.setMidiInputDeviceEnabled(midiInputList[listIndex].identifier, true);
        deviceManager.addMidiInputDeviceCallback("", &midiInputCollector);
    }


    void disableAllMidiOutputs()
    {
        if (midiOutputDevice == nullptr)
            return;

        midiOutputDevice->clearAllPendingMessages();
        midiOutputDevice->stopBackgroundThread();
        midiOutputDevice.reset();
    }


    void midiOutputChanged()
    {
        disableAllMidiOutputs();

        int itemId = midiOutputSelection.getSelectedId();
        jassert(itemId != 0);        // Ensure our test has a valid comboBox item selected
        int listIndex = itemId - 1;

        auto midiOutputList = juce::MidiOutput::getAvailableDevices();

        midiOutputDevice = juce::MidiOutput::openDevice(midiOutputList[listIndex].identifier);
        jassert(midiOutputDevice);

        midiOutputDevice->startBackgroundThread();
    }


    void paint (Graphics& g) override
    {
        g.fillAll(getLookAndFeel().findColour (ResizableWindow::backgroundColourId));
    }


    void createGUI()
    {
        //
        // MIDI Inputs
        //
        auto midiInputList = juce::MidiInput::getAvailableDevices();
        int index = 1;

        for (auto x : midiInputList)
            midiInputSelection.addItem(x.name, index++);

        midiInputSelection.onChange = [this] { midiInputChanged(); };
        midiInputSelection.setSelectedId(1);

        midiInputLabel.setText("MIDI Input", NotificationType::dontSendNotification);

        addAndMakeVisible(midiInputSelection);
        addAndMakeVisible(midiInputLabel);

        //
        // MIDI Output
        //
        auto midiOutputList = juce::MidiOutput::getAvailableDevices();
        index = 1;

        for (auto x : midiOutputList)
            midiOutputSelection.addItem(x.name, index++);

        midiOutputSelection.onChange = [this] { midiOutputChanged(); };
        midiOutputSelection.setSelectedId(1);

        midiOutputLabel.setText("MIDI Output", NotificationType::dontSendNotification);

        addAndMakeVisible(midiOutputSelection);
        addAndMakeVisible(midiOutputLabel);

        setSize(400, 200);
    }


    void resized() override
    {
        midiInputSelection.setBounds(10, 30, 300, 30);
        midiInputLabel.setBounds(10, 0, 300, 30);

        midiOutputSelection.setBounds(400, 30, 300, 30);
        midiOutputLabel.setBounds(400, 0, 300, 30);
    }


    void prepareToPlay(int samplesPerBlockExpected, double newSampleRate) override
    {
        sampleRate = newSampleRate;
        expectedSamplesPerBlock = samplesPerBlockExpected;

        midiInputCollector.reset(newSampleRate);
    }


    void getNextAudioBlock(const AudioSourceChannelInfo& bufferToFill) override
    {
        const double current_time = Time::getMillisecondCounterHiRes();

        // Clear audio data
        bufferToFill.clearActiveBufferRegion();

        // Get MIDI input data
        midiInputBuffer.clear();
        midiInputCollector.removeNextBlockOfMessages(midiInputBuffer, expectedSamplesPerBlock);

        // Create MIDI output data
        midiOutBuffer.clear(); 

        for (auto x : midiInputBuffer)
        {
            auto message = x.getMessage();

            // Forward only note on/off messages
            if (!message.isNoteOnOrOff())
                continue;

            midiOutBuffer.addEvent(message, 0);
        }

        if (midiOutputDevice)
            midiOutputDevice->sendBlockOfMessages(midiOutBuffer, current_time, sampleRate);
    }


    void releaseResources() override
    {
    }

private:
    double                      sampleRate = 0.0;
    int                         expectedSamplesPerBlock = 0;

    // MIDI Inputs
    juce::ComboBox              midiInputSelection;
    juce::Label                 midiInputLabel;

    juce::MidiMessageCollector  midiInputCollector;
    juce::MidiBuffer            midiInputBuffer;

    // MIDI Outputs
    juce::ComboBox              midiOutputSelection;
    juce::Label                 midiOutputLabel;

    std::unique_ptr<MidiOutput> midiOutputDevice;
    juce::MidiBuffer            midiOutBuffer;

    JUCE_DECLARE_NON_COPYABLE_WITH_LEAK_DETECTOR (AudioAppDemo)
};
8

I am convinced the bug is not in my code but in JUCE.

Please forward the above piece of code / zip file to one of your coders responsible for the MIDI implementation on Windows and let them have a look at it. I’m on a tight schedule and wish to get this bug fixed soon.

9

Thanks for providing some example code, that makes it much easier for us to investigate the problem. I can reproduce the issue with the provided sample.

The docs for sendBlockOfMessages say that the time is specified using the same time base as Time::getMillisecondCounter. If I switch your demo to use that function in getNextAudioBlock instead of the HiRes version, then the timing issue seems to disappear and notes are immediately forwarded to the output.

I tried artificially creating very rapid outgoing midi events and didn’t see this ‘burst’ behaviour. My output device is the computer’s built-in MIDI synth.

How large is your audio block size? In your example, you’re sending all of the outgoing messages at time ‘0’. If your blocksize is large (e.g. 4000 samples) then the messages will only be sent once every 100ms or so. Maybe this is the cause of the ‘bursts’ of notes. You might get better results by keeping track of the relative times of the incoming messages, rather than setting them all to 0. It’s probably also a good idea to check that the outgoing note-ons and note-offs are properly matched (i.e. avoid sending two consecutive note-ons on the same key) just in case this is confusing the output device somehow. You could also try sending the outgoing midi events to a MIDI Monitor app so that you can check the message order and timestamps yourself. If the messages look good in a MIDI monitor, then the ‘burst’ issue is probably caused by the MIDI output device, rather than the app itself.

This is the intended behaviour. The demo isn’t intended to pass messages from inputs through to outputs. It just logs incoming messages, and allows messages to be sent from the onscreen keyboard.

1 Like
10

Thank you! That would explain the odd behaviour. I took the high resolution time counter function from JUCE tutorial/example code (can’t remember which one).

The MIDI burst effect happened (as mentioned earlier) when I tried changing the JUCE’s timer function to the high resolution one in the MIDI handling area of JUCE code. Just to see if that would fix things and as I was worried that if the high resolution time counter wasn’t used, that would drop the precision of the MIDI timing to audible levels. Obviously that’s not the case, as with the simple fix of switching the used time counter to the low resolution one works perfectly now.

In the actual application I’m calculating the MIDI message times sample accurately. So that should keep the MIDI messages timed correctly.

Question:
I assume that when I’m using the MidiOutput::sendBlockOfMessages(), that naturally ties the MIDI latency to audio buffer latency and buffer size? So I would better add audio buffer size selection to my application?

Alternatively what is a safe assumption that every computer has as their supported buffer size? I’m thinking of using something like 256 sample buffers for my application. If that would be supported by all platforms, I could remove “unnecessary” audio interface configuration GUI elements from my application.

11

If sendBlockOfMessages is being called from the audio callback, then the size of the audio buffer will have an effect on latency. You could either let the user configure the buffer size, or use the smallest available buffer size that’s greater or equal to some specific value (like 256).

1 Like
12

Seems to work. Thank you! Much appreciated!

13

@reuk - reading this just makes me want to ask:
Are you supposed to call sendBlockOfMessages with Time::getMillisecondCounter or Time::getMillisecondCounterHiRes?

My app/plugins are still under development, but I’ve been calling it with the Time::getMillisecondCounterHiRes. This discussion here is a bit confusing to me. Thanks.

14

My impression from the docs is that getMillisecondCounter is the correct one to use.

15

@reuk - I note that in juce_AudioProcessorPlayer.cpp it uses Time::getMillisecondCounterHiRes:

//==============================================================================
void AudioProcessorPlayer::audioDeviceIOCallback (const float** const inputChannelData,
                                                  const int numInputChannels,
                                                  float** const outputChannelData,
                                                  const int numOutputChannels,
                                                  const int numSamples)
{
    const ScopedLock sl (lock);

    // These should have been prepared by audioDeviceAboutToStart()...
    jassert (sampleRate > 0 && blockSize > 0);

    incomingMidi.clear();
    messageCollector.removeNextBlockOfMessages (incomingMidi, numSamples);

    initialiseIoBuffers ({ inputChannelData,  numInputChannels },
                         { outputChannelData, numOutputChannels },
                         numSamples,
                         actualProcessorChannels.ins,
                         actualProcessorChannels.outs,
                         tempBuffer,
                         channels);

    const auto totalNumChannels = jmax (actualProcessorChannels.ins, actualProcessorChannels.outs);
    AudioBuffer<float> buffer (channels.data(), (int) totalNumChannels, numSamples);

    if (processor != nullptr)
    {
        // The processor should be prepared to deal with the same number of output channels
        // as our output device.
        jassert (processor->isMidiEffect() || numOutputChannels == actualProcessorChannels.outs);

        const ScopedLock sl2 (processor->getCallbackLock());

        if (! processor->isSuspended())
        {
            if (processor->isUsingDoublePrecision())
            {
                conversionBuffer.makeCopyOf (buffer, true);
                processor->processBlock (conversionBuffer, incomingMidi);
                buffer.makeCopyOf (conversionBuffer, true);
            }
            else
            {
                processor->processBlock (buffer, incomingMidi);
            }

            if (midiOutput != nullptr)
            {
                if (midiOutput->isBackgroundThreadRunning())
                {
                    midiOutput->sendBlockOfMessages (incomingMidi,
                                                     Time::getMillisecondCounterHiRes(),
                                                     sampleRate);
                }
                else
                {
                    midiOutput->sendBlockOfMessagesNow (incomingMidi);
                }
            }

            return;
        }
    }

    for (int i = 0; i < numOutputChannels; ++i)
        FloatVectorOperations::clear (outputChannelData[i], numSamples);
}

EDIT: I can’t actually believe you should use Time::getMillisecondCounter() as that’s noted to be inaccurate.

“It should be accurate to within a few millisecs, depending on platform, hardware, etc.”

A few milliseconds in note timing is not a good thing.

1 Like
16

I would also prefer if the high resolution counter was used for all MIDI timing.

17

There are some gotchas when it comes to high resolution timers on Windows, alas - its not as simple as it is on Linux or MacOS, unfortunately.

Some good details from Microsoft themselves on the issues. Key thing is to choose the right timer for the job: QueryPerformanceCounter

18

hello, I am a seasoned beginner of JUCE and recently investigating Android App. I built the code on Windows 11 and it works well. In the case of Android (6.0), however, it crashes perhaps because Android has no default midiout device whereas Windows has “Microsoft GS wavetable synth”. So I added the following conditional statement:

    // 
    // MIDI Output
    //

auto midiOutputList = juce::MidiOutput::getAvailableDevices();
    index = 1;

**if(midiOutputList.size() != 0)**
  {
    for (auto x : midiOutputList)
      midiOutputSelection.addItem(x.name, index++);

    midiOutputSelection.onChange = [this] { midiOutputChanged(); };
    midiOutputSelection.setSelectedId(1);
  }

It will safely launches but there is no midi device in the combobox (both midiin and midiout) in spite of plugging ROLI seaboard block and M-Audio midiman.

One trivial general question about Android MIDI:
Is it possible to use midiout on Android?
As to midiin, I know that it works well.
Thanks

19

Of course!

You could try building the demorunner for Android, testing if MIDI output works properly and then looking under the hood.

20

Thanks for advice.
I built DemoRunner for Android 6.0 ( physical).
The result:

the launch succeeds but it represents no widget. So I tried to build “MidiDemo.jucer”(one of examples), but it unfortunately crashes.

my environment:
JUCE module 7.0.2
Android Studio Chipmunk 2021.2.1 patch 2
Android: alcatel idol4 (6.0.1)

21

Hi,
I’ve found this post, which is helpful and has solved the issue when building “DemoRunner” for old Android OS .
I’ve confirmed that it succeeds to build “DemoRunner” for Android 6.0 and MIDI IN-OUT works well.
Thanks!

1 Like