I just had a look at the MidiInput code on Windows. Might be a stupid question, but why is there actually a extra thread? Why not pass the MIDIMessage directly to the MidiInputCallback::handleIncomingMidiMessage() instead of doing it via an extra thread? The only reason I can imagine of is that you wanted to avoid the situation where slow user-code in MidiInputCallback::handleIncomingMidiMessage() might block the OS for too long.
Since the message-collecting thread is only running at a slightly than higher priority of 6, I could imagine that some MIDIMessages would only reach MidiInputCallback::handleIncomingMidiMessage() some ms after they have come in (depending on how many threads are concurrently running at which priority). This means added latency, even jitter which could have been avoided.
It’s quite a few years since I wrote that, so I can’t remember exactly, but there were definitely problems if that thread was blocked for too long. Yes, it’ll add a (very very small) latency, but the important thing is that the timestamps on the events will be correct, and any jitter-sensitive calculations should be based on that, not on the time that it arrives at the callback.
What you’re saying is true only for recording the incoming MIDI data, but not for realtime VSTi playback, because if the correctly timestamped MIDI data is only passed 2 or more buffers later on to the ASIO processing instead of the buffer directly after the previous one, then you end up with a bad MIDI timing when playing your VSTi’s - although, as you suggested, when recording this events and playing them back later on, the timing will be correct. If the MIDIInput class would directly forward the MIDIMessage’s, this problem would not arise. I’m now using my own classes, but I’m sure JUCE users would be happy to see a option that allows excluding this thread-based mechanism for MIDI Input - hence offering a much higher precision for playing VSTi’s.
Oh, it won’t be delayed by that much! I’ve not measured it, but I’d be surprised if it took more than a couple of millisecs to wake the thread up. It’d be interesting to add some tests in there to find out.
Actually, I think I remember the reason I did it now: win32 had some severe restrictions on what you’re allowed to do inside that callback - no locks, no memory allocations, etc., and I was getting deadlocks. Strangely, looking at their docs now:
…it only says you should avoid other MM calls, so maybe in later OS versions they improved their own threading model.
64 samples ASIO latency = 1.4 ms is about less than “a couple of milliseconds” . it’s obvious that for such short latency you’ll not have the midi data ready for every next buffer processing using this threaded approach, especially under heavy CPU load.
I also wanted to point out that the MidiMessageCollector uses Time::getMillisecondCounter() which I found to be as inaccurate as 16ms (!) on my Win XP 32 system. It reports 0 for a few times, then 16, then again 0 for a few times, etc…
Wow - that must be a creaky old system! The library always sets timeBeginPeriod (1) so it really should be returning 1ms accuracy.
But I just checked, and MidiMessageCollector actually uses getMillisecondCounterHiRes() - are you not using a recent version?
You’re right, I still had an old MidiMessageCollector code here that uses Time::getMillisecondCounter(). Didn’t check the tip. Sorry for that.
Are you sure timeBeginPeriod(1) does actually apply to juce_millisecondsSinceStartup()'s GetTickCount() ? I thought it only applied to timeGetTime(), which is by the way apparently better than GetTickCount() ?
That code’s been there for years, I can’t even remember writing it! Well, timeGetTime is more accurate, but GetTickCount is very fast… I guess these days the speed is less important than it used to be, so yes, maybe it’s a good idea to change to timeGetTime.
BTW, here’s an old discussion:
http://www.rawmaterialsoftware.com/viewtopic.php?f=3&t=1767&hilit=gettickcount
Thanks, I actually read it before posting this 
I’m now trying to find out how accurate I can get my sequencer’s MIDI I/O. I’ll certainly post the results. I’ve been disappointed by most sequencers so far, only very few seem to be tight (at least on Windows).
Hey Jules, I’ve made the tests (on Windows XP)…
First of all, I’ve completely removed all threading stuff from the MidiInput. Previously there would be a jitter of around 172 samples on the midi events @ 64 samples buffer size, 44.1kHz. This was with the original MidiInput & MidiInputCollector code.
After removing the thread in MidiInput, and passing the midi events over directly, the accuracy would most of the time be around 20 samples, but sometimes go back to 172. I don’t know why…
So I wrote my own MidiMessageCollector which uses absolutely no CriticalSection, but a non-locked FIFO based Midi input buffer. The result is unbelievable: I get a jitter of only 20 samples all the time now ! That’s an accuracy that’s better than 1ms. I’ve tested other sequencers like Ableton, Cubase and FLStudio and they all have a jitter of around 170 samples, which one can hear. It’s even better than my Akai S3000XL sampler! I’ve just routed the MIDI Input to a VSTi that is sample-accurate and which makes a “blip” upon note on’s. I recorded the audio output and checked the delta times between the blips. The more the same they are, the less the jitter.
You might ask yourself what’s my reference MIDI signal and how it can be so tight that I can make such conclusions. Well, it’s an old hardware step sequencer based on a 8052, written in assembler, which I built a few years ago. The timing of this sequencer is as good as MIDI allows. I just let run the sequencer at around 200 bpm and output 1/16 notes all the time. It’s MIDI output is connected to my Delta1010LT’s MIDI input.
If you want I can send you the audio samples.
Nice work!
[quote=“zamrate”]I just had a look at the MidiInput code on Windows. Might be a stupid question, but why is there actually a extra thread? Why not pass the MIDIMessage directly to the MidiInputCallback::handleIncomingMidiMessage() instead of doing it via an extra thread? The only reason I can imagine of is that you wanted to avoid the situation where slow user-code in MidiInputCallback::handleIncomingMidiMessage() might block the OS for too long.
Since the message-collecting thread is only running at a slightly than higher priority of 6, I could imagine that some MIDIMessages would only reach MidiInputCallback::handleIncomingMidiMessage() some ms after they have come in (depending on how many threads are concurrently running at which priority). This means added latency, even jitter which could have been avoided.[/quote]
er… maybe I’m missing something here, but aren’t the keywords: “optional”, “thread”, and to a lesser extent “slow user code”?
I can see how this might sense in a VST environment in an attempt to reduce latency while using handleIncomingMidiMessage, but probably not when attempting to handle large sysex files via handlePARTIALsysexmessage (large is of course implied by the word “partial”)… and it’s just not practical in these situations where one might be handling large amounts of MIDI data from multiple devices to lock up the messaging thread with callbacks and potentially dropping packets due to “slow user code”.
I’ve only just merged in the tip (1.52), so I haven’t had a chance to dig into the new MIDI stuff yet, but before I go re-introducing an “optional” thread to MidiInputCallback, is there a better way to do this? Perhaps the handler itself should be a thread? What’s the best approach here.
Is there some documentation/forum thread which better explains what and why the changes are? I do see that handlePartialSysexMessage is now handled the same way on all platforms, but I already fixed that stuff ages ago 
Thanks
I think the idea here is to reduce latency when getting Midi Messages directly from Midi Controllers, not when reading files. Obviously you’d still use a thread when reading from an asynchronous source like a file, just like when you read from an audio file. But here you just enqueue the message and process it somewhere else if you want.
And I don’t think the changes have been precisely communicated on the forum, because I’m quite interested in them so I’ve been watching
zamrate,
does your code address situations like bigger incoming sysex messages? Would you mind sharing the code?
+1 for sharing the code. Or even integrating it to juce ? Jules ?
I integrated something like this months ago.
Great ! 