I’ve noticed some quirky multi-touch behaviour and would be grateful for any feedback. For reference, I’m on Windows 10 and it’s the same for both standalone apps and VST plugins. I don’t know whether it’s platform-specific or a general thing at this stage.
Single touch and mouse inputs work very nicely side by side, but as soon as multi-touch input is introduced, the mouse input recognition stops altogether (clicks, hovers and so on do nothing), while single touches still work as before. Is there something I’m doing wrong, or could this be improved?
After reading various threads here on the topic, I’ve come to appreciate that Windows generally treats single touch and mouse events as basically equivalent but then things change with multi-touch, so I imagine it’s some funky statefulness in this area, such as a one-way switch being thrown (no) or a blocking list of active touches (yes, see below) that isn’t ever fully cleared?
Update: after some quick digging, I’ve found some details that seem to confirm my hypothesis:
areOtherTouchSourcesActive() in juce_win32_Windowing consistently returns true (incorrectly) after any multi-touch events have occurred, which blocks any subsequent mouse events from firing normally thereafter
it seems that Desktop::getInstance().getMouseSources() increases from the initial 1 to 1 + maximum number of simultaneous touches received; this number never decreases
isDragging() (aka buttonState.isAnyMouseButtonDown()) always returns true for these extra mouse sources, except for the final one which is duly cleared to false
So it would appear that a proper inventory and suitable clearing of all “mouse sources” is what’s required here. I’ll keep looking into it.
The good news is that the mouse input isn’t being ignored any more. Definite progress, thank you!
The slightly unfortunate news is that now one touch event out of every touch group is being ignored (it seems to be the last one): i.e. a single touch produces no touch, two touches together only produce one, three touches produce two, etc.
Sorry, I’m back with some more multi-touch shenanigans.
I have two further questions/observations:
Hoverculosis
Every multi-touch group leaves behind a hover over the last item touched, and these hovers often remain even after further multi-touching. You can mop them all up with judicious mouse meanderings afterwards; while Hong Kong Phooey would no doubt enjoy this activity, I’m sure some users wouldn’t want a bar of it.
I’m wondering whether we could/should completely remove the hover artefact from these touch situations (my thinking at this stage), or at least limit it to the very last one in all cases? But even when a single hover remains, it isn’t cleared by the next mouse hover, so that feels slightly ungainly. Rolling, “arpeggiated” events don’t suffer as badly as short, sharp “clusters”. Thoughts?
Touch and go
I’m experiencing missing touch events when input is split between multiple instances: e.g. two VST plugin windows on the same screen, a triple touch on one and a triple touch on another. In such a situation, one of these windows will always respond perfectly with three events (the slightly earlier one, I think), while the other will do anything from zero to two, but never all three. This is also the case with simple single touch events, with only one of the pair being processed. I haven’t investigated this in detail yet, but any ideas on whether it’s:
a simple fix (fingers crossed)
a concurrency issue of some kind (I’ve been using multiple instances of the same plugin to test because it’s the only one I have with a touch-obvious GUI, whether that makes a difference), or
a baked-in OS limitation (cue sad face and poor quality virtual violin samples).
Luckily, the last of these seems somewhat unlikely, as all touch events are appearing on the screen as happy little circles.
If possible, I would love to be able to support multi-window, multi-app, multi-user collaborative multi-touch without multi-headaches. Thanks for any help in this!
I’ve cleaned up the Windows touch input code a bit which seems to have solved the hover issue. I can’t reproduce the second issue of missing touch events though - if you could send some code to reproduce it I’ll take a look.
The recent hover fixes are absolutely gorgeous; now everything is working really well in this regard. Thanks @ed95!
After building some more multi-touch plugins, I’ve narrowed down the problem to multiple groups of multi-touch events when running multiple instances of the same VST dll within a single host and not using dedicated process bridging. Here is my touching story so far:
standalone app and plugin: WORKING
2 instances of the same standalone app: WORKING
2 instances of the same plugin in different DAWs: WORKING*
2 instances of different plugins in the same DAW: WORKING
2 instances of the same plugin (each in a dedicated process) in the same DAW: WORKING
2 instances of the same plugin (separate process bridging or native hosting) in the same DAW: FAILING
I’ve tested these cases in both Reaper and FL Studio and the results are consistent, so it’s looking a lot like a concurrency issue in which the two instances are subtly interfering with each other on some level. Does this happen in the general case with JUCE-based VST plugins or is it an ijijn-specific problem? Also, for this to happen at all I guess this means they must be sharing some relevant state information between them somehow. Any pointers?
For now I’ll get to work reducing an example plugin to the essentials to see how I get on.
*Perhaps of interest is that there is often a considerable delay (half a second or so) between the two rounds of nearly simultaneous events firing in this scenario, although the final results are entirely as expected. It may be a simple question of finite resources between competing host processes, but this does seem particularly drawn out compared to the otherwise responsive experience and I’ll note it here anyway for completeness.
Copy VST dll to host’s plugin folder or point host to build folder
Load two instances of the plugin in the host, making sure they’re not bridged in dedicated processes (this is unlikely by default)
Arrange the plugin windows so that you can see both at once
Touch and hold multiple buttons in both windows at the same time
You can clearly see which ones are working and which ones are being ignored or triggered prematurely. In a 3 on 3 touch-of-war I’ll typically get either 3-0 or 2-1, one way or the other. Also, touches in one will steal from held multi-touch groups in the other, activate the stolen group’s “down” events unexpectedly, before failing to fire off anything at the end of it all.
