Mouse data to MIDI

General JUCE discussion
1

Hi, I’m a casual programmer so this thread might seem stupid.

I had an idea to make a desktop program that would listen for a specific hotkey to be held down. When that condition is met, it would make the mouse cursor disappear and use some processing to generate MIDI CC messages based on its movement speed. Hopefully it feel similar to a stringed instrument bow if done right. You would be able to find this program in your DAW’s MIDI input devices and record it on a track.

I’m a bit confused where to do this processing though, since no audio is actually processed. Should I be doing stuff in getNextAudioBlock()?

When I try to do something simple like just read the location of the mouse on the screen and print it, it only updates when the app first starts up.

2

read up on MouseListener methods and how to override them.

3

When I read about MouseListeners, it seems like they are designed more for mouse functions performed on or within a component. I’m more after tracking general mouse position when a hotkey is pressed.

I was thinking of using Desktop::getInstance().getMainMouseSource().getScreenPosition().getX() and Desktop::getMainMouseSource().enableUnboundedMouseMovement() to achieve the mouse behavior I want. I’m just unsure of where exactly to be calling these.

4

I read some info that it isn’t possible to create a virtual MIDI output on Windows, so it looks like I am going to be making a plugin version of this.

Is there a way to get the current sample in the buffer? I want to process faster than just every block, and I’m not sure how to get output from my mouse processor into a buffered format.

5

On Windows you’d normally use one of these:

https://www.tobias-erichsen.de/software/loopmidi.html

Then send your MIDI events to the output of the virtual MIDI port and activate the input in your DAW. On macOS you can create MIDI ports easily.

I did something similar (mouse to MIDI for a bow simulation) a few years ago. I ended up using a HighResolutionTimer with a frequency of 15 - 20 ms and the MidiOutput::sendMessageNow (or something like that, don’t remember the exact API name) inside this callback. Since your data source is the UI thread, you won’t get a predictable frequency for the mouse refresh events, but you need to filter (smooth) the mouse data anyway because otherwise you’ll get a lot of jitter.

Also I wouldn’t bother about inter-buffer accuracy, the mouse events come in at a frequency of 30ms max.

1 Like
6

Thanks for the info. I have a gaming mouse which polls at 1000 Hz, so if the mouse events followed that it would be much faster than every 30 ms. Not sure if the events come in as fast as the polling rate though.

Do you have the code for that project somewhere I could look? It might help me out.

7

I’m also having problems getting Desktop::getInstance().getMainMouseSource().enableUnboundedMouseMovement(true, false); to work.

When I try to call it nothing seems to happen. Does anyone have any advice?

8

What do you expect to happen? How did you test this?

I’d put that line in mouseDown() and print in mouseDrag() the position.
Expected behaviour: global positions can become negative and higher than the width and height…
Without, I’d expect the global positions to be restricted to the screen coordinates.

Is this what you tried?

HTH

Addition: and in the mouseDown you don’t use the Desktop, you can specifically use the MouseSource sending:

mouseDown (const MouseEvent& e) override
{
    e.source.enableUnboundedMouseMovement (true, false);
    // ...
}
9

I’m not wanting to use mouseDown(). I want to be able to push a keyboard hotkey that toggles unbounded mouse movement on and off.

So, say you push the tilde key, the mouse cursor then disappears, can move infinitely, and the plugin outputs MIDI data based on the movement speed of your mouse. When you push the tilde key again, it brings the mouse cursor back and disables the midi output.

10

I see, in that case you picked the wrong method. The enableUnboundedMouseMovement() is meant for mouse drags, as it is written in the docs.

You can achieve what you want by adding a global mouse listener to the desktop:
Desktop::addGlobalMouseListener()

Inherit your class from MouseListener (note that all Components are already mouseListeners). When your magic key is pressed, call Desktop::getInstance().addGlobalMouseListener (this); and on deactivate Desktop::getInstance().removeGlobalMouseListener (this);

11

Thanks for the advice. I’ll try this out tomorrow and post back with results!

12

Just tried this. It doesn’t seem to do anything. I think maybe my problem is misunderstood.

I am already able to read the mouse position on the screen and use it properly. I just want to be able to push a hotkey that hides the cursor and gives it unbounded movement. Ideally this would not require pushing the mouse button.

I am also unable to get any key other than the space key to do anything. Most other keys make the Windows error message sound, and the keys that don’t just don’t do anything.

13

If you don’t share any of your code, we can’t really help you figure out why it doesn’t do what you want or expect it to do… 3 backticks on the line before and after any shared code will format it like this:

//here's a comment
aStatement(*this);

the backtick is on the same key as the ~ key.

14

Here is what I’m doing:

bool MidimouseBowAudioProcessorEditor::keyStateChanged(bool isKeyDown, Component *originatingComponent)
{
    // If this is anything other than the space key it doesn't work
    if(KeyPress::isKeyCurrentlyDown(KeyPress::spaceKey))
    {
        if (!_mouseIsUnbound)
        {
            // Neither of these do anything
            Desktop::getInstance().getMainMouseSource().enableUnboundedMouseMovement(true, false);
            Desktop::getInstance().addGlobalMouseListener(this);

            _mouseIsUnbound = true;
        }
        else if (_mouseIsUnbound)
        {
            // Neither of these do anything
            Desktop::getInstance().getMainMouseSource().enableUnboundedMouseMovement(false, false);
            Desktop::getInstance().removeGlobalMouseListener(this);

            _mouseIsUnbound = false;
        }
    }

    return false;
}

I have verified that the code is working properly. If I put a test variable in the inner if statements and print it, I can tell the if statements are being triggered as intended. The problem is none of the methods will make the mouse cursor disappear and become unbound by the extremities of the screen.

Keep in mind that currently I am working with VST, not a standalone app.

15

IIRC I was putting the mouse cursor in the middle of the screen and just use the “derivation” as movement speed back then:

// terrible pseudo code approaching...
void mouseMove(const MouseEvent& e)
{
    if(!active)
        return;

    const int speed = e.getMouseX() - midPoint;
    Mouse::setPosition(middleOfScreen);
}

This way you don’t need unbound movement because you have to be really fast to hit the edge of the screen.

I’d share the code but I can’t find it anymore. It was also one of the first things that I wrote in C++ so people would just laugh at me…

16

That works. Setting the mouse position seems to be one of the few things you can do to the global mouse source with results.

Unfortunately, I realized that in order to get a key binding to work, you have to click on the window to get focus. This would be extremely annoying with what I would want to do. It doesn’t seem like I can get this to work out properly.

17

I used MIDI input for activating the “mouse bow mode” to get around this:

  • if a note is pressed down, I set the mouse position to the middle (saved the previous position for an awesome UX) and activated the measurement.
  • if the last note is released, the mouse position is set to the previously stored position and the measurement is stopped.
18

That’s a cool solution. I unfortunately don’t use a MIDI keyboard however. I suppose I may give it a try sometime in the future.