Live 9 Windows crash (clicking into UI while pressing key)

#1

The problem is easy to reproduce (i used Windows 8 and ableton 9 Demo) and its related to the keyboard hook fix for ableton live:

Load a VSTi Synth and make sure it has the focus. Then press a Keyboard key that LIVE consumes (for example ‘e’) and keep it. Now click somewhere into the plugin UI. After that the LIVE UI and the plugin does note response to mouse events anymore. The only thing you can do is to quit live with a task kill.

The debugger shows me that the program spends a lot of time in the keyboardHookCallback method while the UI does not response anymore:

    LRESULT CALLBACK keyboardHookCallback (int nCode, WPARAM wParam, LPARAM lParam)
    {
        if (nCode == 0)
        {
            const MSG& msg = *(const MSG*) lParam;

            if (msg.message == WM_CHAR)
            {
                Desktop& desktop = Desktop::getInstance();
                HWND focused = GetFocus();

                for (int i = desktop.getNumComponents(); --i >= 0;)
                {
                    if ((HWND) desktop.getComponent (i)->getWindowHandle() == focused)
                    {
                        SendMessage (focused, msg.message, msg.wParam, msg.lParam);
                        break;
                    }
                }
            }
        }

        return CallNextHookEx (mouseWheelHook, nCode, wParam, lParam);
    }

This fixes the problem:

            //if (PluginHostType().isAbletonLive())
                //registerKeyboardHook();

Related thread (VST Plugin Still Not Getting Keystrokes):
http://rawmaterialsoftware.com/viewtopic.php?f=8&t=6901

#2

Ah… it should probably be using PostMessage instead of SendMessage.

#3

Thanks for looking at that. PostMessage does not fix the problem. The UI now already blocks when i try to enter a text into a text editor.

#4

Ok, well I don’t know then…

#5

No problem. I cant believe that no one else has this issue :slight_smile:

#6

Well, clicking whilst holding down a letter-key is probably quite an unusual thing to do.

#7

I’m experiencing the same problem: press a key ‘c’ while the juce based plugin has focus and Live will hang. Very dramatic. This happens using the Juce tip and either Live 8.1.4 32bit or 9.0.5 64bit.

The culprit seems to be the SendMessage() call in keyboardHookCallback() in juce_VST_Wrapper.cpp. Every time this is called this leads to at least one more WM_CHAR message received by the hook, causing an infinite loop.
It’s not a code reentrancy problem and switching to PostMessage() does not help either.

The best solution is probably to not register the hook at all for Ableton 8 and 9.
A relatively elegant solution could be to change line 1341 in juce_VST_Wrapper.cpp to this:

#if JucePlugin_EditorRequiresKeyboardFocus
            if (PluginHostType().isAbletonLive())
                registerKeyboardHook();
#endif

The dev can then control whether the hook is installed or not using the IntroJucer parameter “Plugin editor requires keyboard focus”

Of course, without the hook the plugin does no longer receive keystrokes. This can be solved by having the user create an Options.txt file in the Live preferences directory at
c:\Users<user>\AppData\Roaming\Ableton\Live n.n.n\Preferences\

In order for the plugin to receive keystrokes the options file should contain this switch:

-_EnsureKeyMessagesForPlugins

Of course you then lose global keystrokes in Live itself when the plugin has focus, such as space to start/stop the transport. But we cannot have everything in life, can we…

#8

Excellent idea about wrapping that hook in the JucePlugin_EditorRequiresKeyboardFocus macro - I’ll get that sorted out. And thanks for the clear explanation of Live’s setting files too!

#9

great, thanks Jules

btw while at it, you might perhaps remove the whole keyboard hook as well, since it has no real added value any more because of the way Live conflicts with it. People can always enable keystrokes with the Live options.txt file if they must have it.
Unless I overlook cases where it really is necessary to have the hook?

#10

TBH I can’t remember all the details, but I certainly wouldn’t have added it for no reason! But if it definitely screws up in Live 8 + 9, then perhaps I should leave it in there just for v7 and earlier?

#11

I only tested on 8 & 9. For all I know Live 7 could have the same problem…
I guess the hook was originally added as a workaround to Live’s default behaviour of not passing keystrokes to plugins? Maybe before there was an Options.txt switch to change that behaviour?

Btw the hook seems to induce an additional problem besides the hang/crash : when displaying a juce dialog window containing an edit box, any keystrokes entered there are duplicated 3 or 4 times.

#12

Hmm, it does sound problematic. Doing a search, it seems that someone sent me the hook code back in 2009. I guess that unless anyone has an objection, I should just get rid of it?

#13

It might be wise.
I had to remove this code as well following some report of issues.

#14

Righto. Will expunge it…