Can WebBrowserComponent forward key presses to the DAW?

Audio Plugins
1

Running into an issue using the WebBrowserComponent as the GUI for my plugin. Whenever the part of my GUI that is a WebBrowserComponent is in focus, all key events get “swallowed” up. This means that if any controls are moved within the web browser and for instance the user presses spacebar in a DAW, the DAW’s transport doesn’t start. (It does if a non-WebBrowserComponent part of the GUI is in focus). Same idea with any DAW hotkeys that are pressed while the WebBrowser is in focus. Is there any way that this issue can be resolved in JUCE?

We have worked around the issue on Mac with a series of hacks for injecting Javascript into the web browser to get key events and then forward them onto the app using NSApp performKeyEquivalent, but have not found a similar hack to work for Windows, as it does not appear possible to send keystrokes to a window unless it is in the foreground using SendInput. (Though maybe a Windows expert can tell me if I’m wrong on that) And would be ideal to avoid hacks like this in the first place if possible.

1 Like
2

Hi jackwcampbell i stumbled accross the same problem. Did you find any solution to this?

3

Unfortunately nothing in JUCE itself – it would be awesome if JUCE would address this issue because it really makes using the WebBrowserComponent for audio plugins infeasible/clunky in many cases. (@tpoole ?) Users generally expect for instance to be able to start/stop the transport with spacebar regardless of whether a plug GUI is in focus…

We ended up ditching the WebBrowserComponent for now and writing our own since our mods to fix this diverged so much from the mainline JUCE component, but we did find a solution at least for most DAWs (except Logic M1 because of its subprocess). This approach can be hacked into the JUCE WebBrowser or incorporated into your own if you prefer. You can inject JS into the web page that you load which listens to all key events on the web view and sets a flag if the event should be forwarded to the DAW. Then in your WebBrowserComponent, you set a hook on the native window to get notifications on all key-downs (this has to be done on the native window, not the JUCE component, because the JUCE component doesn’t seem to get the key down when the web browser is in focus…). Then call your injected Javascript to see if its “forward event” flag is true – if so, forward the event to the host window. On Mac this is done by switching the first responder to the host window and calling keyDown on it. On Windows, this is done by switching the host window to be in the foreground and using SendInput to send the keystroke to the host window.

Looking around on the web, this is the common way around this issue (usually used with Microsoft WebView2 but works with WkWebView on Mac as well). It would be better if JUCE handled this, because it has knowledge of native windows etc and is unfortunate for the client code to have to handle it…

3 Likes
4

And FWIW, Microsoft is planning on adding a true key forwarding/detecting callback to a future WebView2. Currently they only support forwarding accelerator keys: WPF Key Forwarding from WebView2 to Host application? · Issue #468 · MicrosoftEdge/WebView2Feedback · GitHub

AFAIK, the WebKit team is not planning a similar fix so the Javascript hack is the way that folks are handling it… which sucks.

(So, not to post all the blame on JUCE – but there is a way that JUCE could handle it, it’s just not trivial unfortunately.)

1 Like
5

Thanks so much for the detailed answer! Currently working on the windows fix and I have already cobbled together a communication between the plugin and the WebView2 for other purposes so I can send events to the plugin and fake the key input from there.

What I do to get the HWND handle to the DAW window (to which I send all necessary key events) feels even more hacky; when a new Plugin instance is opened I save the response of GetForegroundWindow at the very beginning assuming that the DAW is in the foreground when the user opens a plugin. Would you know if there is a better/more robust way to obtain that handle?

6

One thing you can try is to focus your plugin’s JUCE window in response and send the input to that, which should put the key event back into the window hook chain (similar to the responder chain on macOS) and then be forwarded to the DAW by the OS. So rather than grabbing the DAW’s HWND, you get the JUCE window HWND using getPeer()->getNativeHandle(). Then you can use the win32 SetFocus on that HWND and use the win32 SendInput() call to inject the JS key events back into the chain (which should end up then being forwarded to the DAW window as well).

7

If I understood you correctly, that’s what I tried first but here’s the problem: if I send the key event to the JUCE window HWND then it’s consumed by the WebView2 again (essentially creating a feedback loop) or am I missing something?

8

I think the key is to switch focus to the JUCE window HWND first and then use SendInput, because in that case the WebView2 won’t be in the chain for receiving events (should be skipped, which avoids the feedback loop). Though I’m a little fuzzy in my memory on that… I did run into feedback loop issues trying a similar approach on Mac, but IIRC it wasn’t an issue on Windows.

9

Just in case someone ever has the same problem: The solution outlined by jack works great on Mac (tested ableton and FL) and on Windows for Ableton. For Windows Fl studio I found with trial and error that I need to take the “grandparent” of the HWND I get with getPeer()->getNativeHandle() (call getParent() on it twice) in order for FL studio to get my spacebar presses to start playback. Thanks again jack for the great help!

1 Like
10

Heya - sorry to bump an old thread, but I’ve just run into the same problem. However, neither Ableton nor FL Studio seem to pick up on the SendInput events properly.
For both DAWs, if a text field is highlighted, the key events will work and type characters - so it seems the events are making their way to the DAW. However, sending a space when a text field isn’t highlighted does nothing - it should start/stop playback.

Here’s how I’m sending the key events:

        INPUT input;
        input.type = INPUT_KEYBOARD;
        input.ki.wVk = 0;
        input.ki.wScan = (unsigned char) ' ';
        input.ki.dwFlags = KEYEVENTF_UNICODE;
        input.ki.time = 0;
        input.ki.dwExtraInfo = 0;
        SendInput(1, &input, sizeof(INPUT));

I’m guessing that either these DAWs have changed how they pick up key events for non-text related keypresses, or I’m doing something wrong while sending the events. @gm1 & @jackwcampbell, did either of you run into this problem?