MidiKeyboardComponent::getNoteAtPosition() not working correctly

General JUCE discussion
1

I’m trying to draw on top of the MidiKeyboardComponent and still have the MKC respond to mouse events and am not having any luck passing those events captured by the opaque component on top of the MKC to the MKC. here’s my simplified code:

struct SeeThruWindow : public Component
{
    SeeThruWindow() {}
    ~SeeThruWindow() {}

    void paint(juce::Graphics &g) override {
        if( mouseMovedOver ) {
            g.setColour(Colours::red);
            //draw a little red dot wherever our cursor is in the window
            g.fillEllipse(mouse.getX()-2.5f, mouse.getY()-2.5f, 5, 5);
        }
    }

    void mouseDown(const MouseEvent& event) override {
        DBG( "SeeThruWindow mouseDown: " + event.getPosition().toString() );
        if( Component* p = getParentComponent() ) {
            p->mouseDown(event.getEventRelativeTo(p));
        }
    }
    void mouseMove(const MouseEvent& event) override {
        if( Component* p = getParentComponent() ) {
            p->mouseMove(event.getEventRelativeTo(p));
        }
        mouse = event.getPosition();
        repaint();
    }
    void mouseEnter(const MouseEvent& event) override { mouseMovedOver = true; }
    void mouseExit( const MouseEvent& event) override { mouseMovedOver = false; mouse = {-5, -5}; repaint(); }

    Point<int> mouse;
    bool mouseMovedOver = false;
};

and now the component that has the SeeThruWindow and the MidiKeyboardComponent as children

class PianoRoll    : public Component
{
public:
    PianoRoll() : keyboard(*state, MidiKeyboardComponent::horizontalKeyboard)
    {
        addAndMakeVisible(keyboard);
        addAndMakeVisible(seeThruWindow);

        keyboard.setScrollButtonsVisible(false);
        keyboard.setOctaveForMiddleC(4);
    }
    ~PianoRoll() {}

    void paint (Graphics& g) override { g.fillAll(Colours::white); }
    void resized() override
    {
        auto r = getLocalBounds();
        keyboard.setKeyWidth(float(r.getWidth()) / float(keyCount) );
        seeThruWindow.setBounds(r);

        FlexBox fb;
        fb.flexDirection = FlexBox::Direction::column;
        float def = 128.f; //25 key = 128.f

        fb.items.add(FlexItem().withFlex((def-108.f) / def) );
        fb.items.add(FlexItem( keyboard ).withFlex(108.f / def ) );
        
        fb.performLayout(r);
    }
    
    void mouseDown(const MouseEvent &event) override
    {
        //we clicked on the SeeThruWindow
        auto kb = keyboard.getBounds();
        if( kb.contains(event.getPosition()) ) {
            DBG( "Keyboard contains event position" );
            auto n = keyboard.getNoteAtPosition(event.getPosition());
            DBG( "key under cursor " + String(n) );
        } else {
            DBG( "keyboard does not contain event position" );
        }
        if( owner != nullptr )
            owner->mouseDown(event);
    }

    void mouseMove(const MouseEvent &event) override
    {
        keyboard.mouseMove(event);
        if( owner != nullptr )
            owner->mouseMove(event);
    }
    void setOwner(Component* _owner) { owner = _owner; }

    /** these are the types of midi controllers we can display, organized by number of WHITE KEYS */
    enum class KeyCount {
        KeyCount88 = 7*7 + 2 + 1,
        KeyCount76 = 7*5 + 5 + 5,
        KeyCount73 = 7*5 + 3 + 5,
        KeyCount61 = 7*5 + 1,
        KeyCount49 = 7*4 + 1,
        KeyCount44 = 7*3 + 1 + 4,
        KeyCount37 = 7*3 + 1,
        KeyCount25 = 7*2 + 1,
    };

    struct KeyRange {
        int lowKey;
        int highKey;
    };

    /** these are the midi note ranges for each type of keyboard */
    std::map<KeyCount, KeyRange> const KeyRanges{
        {KeyCount::KeyCount88, {21, 21+87}},
        {KeyCount::KeyCount76, {60-24-8, 60-24-8+75}},
        {KeyCount::KeyCount73, {60-24-8, 60-24-8+72}},
        {KeyCount::KeyCount61, {60-24, 60-24+60}},
        {KeyCount::KeyCount49, {60-12, 60-12+48}},
        {KeyCount::KeyCount44, {60-19, 60-19+43}},
        {KeyCount::KeyCount37, {60-12, 60-12+36}},
        {KeyCount::KeyCount25, {60-24, 60}}
    };


    /** this is how you set the number of visible keys on screen */
    void setKeyCount( KeyCount k )
    {
        KeyRange const& range = KeyRanges.at(k);
        keyboard.setAvailableRange(range.lowKey, range.highKey);
        keyCount = k;
        resized();
    }

private:
    SharedResourcePointer<MidiKeyboardState> state;

    MidiKeyboardComponent keyboard;
    SeeThruWindow seeThruWindow;

    KeyCount keyCount = KeyCount::KeyCount88;

    Component* owner = nullptr;
    JUCE_DECLARE_NON_COPYABLE_WITH_LEAK_DETECTOR (PianoRoll)
};

and the relevant part of my main.cpp:

class MainWindow    : public DocumentWindow
    {
    public:
        MainWindow (String name)  : DocumentWindow (name,
                                                    Colours::lightgrey,
                                                    DocumentWindow::allButtons)
        {
            setUsingNativeTitleBar (true);
            pianoRollWindow = new PianoRoll();
            pianoRollWindow->setSize(600, 118);

            pianoRollWindow->setKeyCount( PianoRoll::KeyCount::KeyCount25 );
            setContentNonOwned(pianoRollWindow, true);
            DocumentWindow::setResizable(true, true);
            DocumentWindow::setResizeLimits(128, 128,
                                            Desktop::getInstance().getDisplays().getMainDisplay().userArea.getWidth(),
                                            Desktop::getInstance().getDisplays().getMainDisplay().userArea.getHeight());
            centreWithSize (getWidth(), getHeight());
            setVisible (true);
        }
   private:
      ScopedPointer<PianoRoll> pianoRollWindow;
   };

Whenever I click in the SeeThruWindow, the following is output to the console:

SeeThruWindow mouseDown: 329, 99
Keyboard contains event position
key under cursor -1
SeeThruWindow mouseDown: 260, 88
Keyboard contains event position
key under cursor -1
SeeThruWindow mouseDown: 337, 70
Keyboard contains event position
key under cursor -1

So, the mouse events are definitely within the bounds of the MidiKeyboardComponent instance. but for the life of me, I can’t figure out why MidiKeyboardComponent::getNoteAtPosition(Point<int> pos) just will not work with the position I’m passing.

2

am I doing something really dumb in my code that I’m just not understanding?

3

Just as an update, when I stepped thru the line
auto n = keyboard.getNoteAtPosition(event.getPosition());
in the debugger, all of the Juce methods for hit testing or checking if a point is within a component return true. But the OS X methods (when it gets all the way to the native code) return false for checking if a point is within the component. So, I don’t even understand what is the problem here.

4

specifically, this method is where false is returned, with the line isWindowAtPoint() at the end:

bool contains (Point<int> localPos, bool trueIfInAChildWindow) const override
    {
        NSRect viewFrame = [view frame];

        if (! (isPositiveAndBelow (localPos.getX(), (int) viewFrame.size.width)
             && isPositiveAndBelow (localPos.getY(), (int) viewFrame.size.height)))
            return false;

        if (! SystemStats::isRunningInAppExtensionSandbox())
        {
            if (NSWindow* const viewWindow = [view window])
            {
                const NSRect windowFrame = [viewWindow frame];
                const NSPoint windowPoint = [view convertPoint: NSMakePoint (localPos.x, viewFrame.size.height - localPos.y) toView: nil];
                const NSPoint screenPoint = NSMakePoint (windowFrame.origin.x + windowPoint.x,
                                                         windowFrame.origin.y + windowPoint.y);

                if (! isWindowAtPoint (viewWindow, screenPoint)) //<--- this is what fails
                    return false;

            }
        }
5

It’s really dodgy to attempt to pass mouse event calls from one component to another. A better approach would be to setInterceptsMouseEvents (false) on your transparent layer, and if it really needs to get mouse events, attach it as a mouselistener

6

Just guessing, but maybe this method could help you out:
Point< int > Component::getLocalPoint (const Component* sourceComponent, Point< int > pointRelativeToSourceComponent) const

auto pos = keyboard.getLocalPoint (event.eventComponent, event.getPosition());
auto n = keyboard.getNoteAtPosition(pos);

…if you don’t want to follow Jules’ advice for whatever reason…

HTH

@jules: is there some magic, that would do that conversion, if I attach a mouse listener to another component? Just curious…

7

Well no, because a MouseListener isn’t a component, it’s just a virtual base class, so the code that makes the callback can’t convert the coordinates to match a target component that may not exist.

8

Ok, thanks.
I thought, that it was missing an information to do so. Sorry that I was too lazy to look myself :wink:

9

I appreciate the suggestions from both of you guys, @jules and @daniel. However, they are not helping me understand why it is failing.

this passes:

        auto kb = keyboard.getBounds();
        if( kb.contains(event.getPosition()) ) { DBG( "Keyboard contains event position" ); }

So, obviously, if the keyboard bounds contain the event position, then shouldn’t I be able to get the keyboard note under that event position, since it is within the bounds of the keyboard?

10

…but does it solve the issue?
Here is the mapping:

So if it solves your issue, maybe there was a scaling in place for one of the two components?
Either way, it is safer to use the helper to transform into the component’s space.

11

@daniel your method did not solve the issue. I still can’t get the note underneath the cursor, even if the cursor is within the bounds of the midiKeyboardComponent.

12

ok, thanks… was worth a try…

13

Here’s video showing the weird behavior, including resizing the window:

14

just for testing, I changed the keyboard bounds to take up the entire window, just like SeeThruWindow:

    void resized() override
    {
        DBG( "PianoRoll::resized()" );
        auto r = getLocalBounds();
        keyboard.setKeyWidth(float(r.getWidth()) / float(keyCount) );
        seeThruWindow.setBounds(r);
        keyboard.setBounds(r);
    }

It seems to me that MidiKeyboardComponent::getNoteAtPosition should be a private or protected method since you can’t use it from outside of the MidiKeyboardComponent with any sort of confidence.

15

@jules WHY is it “really dodgy”?

16

Because if a component receives mouse moves, up, down, drag etc messages in the wrong order, e.g. if it doesn’t get a “down” before a “drag” or an “enter” before a “move” then things will start to misbehave.

17

ok. If you watch the video, the mouse clicks are clearly happening within the bounds of the MidiKeyboardComponent, so even if I’m not passing the mouseEvent to the MKC, I should still be able to determine which note is under the cursor. do you have any ideas why it is failing?

    static bool isWindowAtPoint (NSWindow* w, NSPoint screenPoint)
    {
       #if defined (MAC_OS_X_VERSION_10_6) && MAC_OS_X_VERSION_MAX_ALLOWED >= MAC_OS_X_VERSION_10_6
        if ([NSWindow respondsToSelector: @selector (windowNumberAtPoint:belowWindowWithWindowNumber:)])
            return [NSWindow windowNumberAtPoint: screenPoint belowWindowWithWindowNumber: 0] == [w windowNumber];
       #endif

        return true;
    }

For whatever reason, the [NSWindow windowNumberAtPoint:] returns false.

18

I’m afraid I can’t answer your question but from what it looks like, you seem to have an overly complicated approach.

If you simply want to draw something on top of the midi keyboard it’d be way simpler to inherit from MidiKeyboardComponent and use paintOverChildren() and maybe overwrite some mouse callbacks like this:

void mouseEnter(const MouseEvent& e)
{
     // do some stuff needed in the inherited class.
     MidiKeyboardComponent::mouseEnter(e);
}
19

I’m doing way more than just drawing on top of it. But all of what I want to do starts with being able to get the note under the cursor.

20

Looking through the MidiKeyboardComponent code, getNoteAtPosition() calls xyToNote() which calls Component::reallyContains() which in the end calls hitTest(). The latter will return false as the MidiKeyboardComponent lies behind your other component.

You’ll need to fiddle with MidiKeyboardComponent’s code to get the getNoteAtPosition() method working for a MCK that’s not in front. You could also inherit from MidiKeyboardComponent and add a further method which ignores whether the component actually can receive mouse events.