[WebView] Drag and drop audio files out of plugin

[blegoff]

Hi everyone,

I’m working on a WebView based project, and want users to be able to drag and drop audio files out of it (ideally dropping them onto a DAW audio track).

I’ve tried multiple solutions on the javascript side using:

source.addEventListener("dragstart", (event) => {
    ...
    event.dataTransfer.setData(...) // tested with DownloadURL and other formats
}

Unfortunately I wasn’t able to find a working implementation using this logic.

I’m wondering if it’s possible to achieve this in combination with the JUCE backend ?

Or maybe use the fact that the juce::WebBrowserComponent is a derived class of juce::Component?

[reuk]

One approach you could try might be:

• In JS, send a callback to the C++ backend that a drag+drop should start.
• In the C++ callback handler, call DragAndDropContainer::performExternalDragDropOfFiles to initiate a drag+drop operation.

If you search the forum for performExternalDragDropOfFiles, you should find other users who are using to facilitate dragging audio files into DAW projects. Therefore, I’m confident that this approach should work, as long as you can find a way to trigger this C++ call from your JS code.

[blegoff]

Thanks for the suggestion @reuk, but unfortunately it’s not working.

I’ve implemented an event listener in the WebView, triggered when the drag event starts in JS:

source.addEventListener("dragstart", (event) => {
	console.log("dragStart");
	window.__JUCE__.backend.emitEvent("dragStart",  {});
})

It’s properly received in the C++ side and it calls the following code:

StringArray files("/Users/blegoff/Downloads/sample.wav");
performExternalDragDropOfFiles(files, false);

But I’m hitting an assertion in juce_Windowing_mac:

[reuk]

Are you passing a sourceComponent argument to performExternalDragDropOfFiles? If not, try passing a pointer to your top-level component.

[blegoff]

I actually tried that two minutes ago.

It allows me to drag the file now, but as soon as I stop dragging (releasing the mouse), it hits the same assert.

[blegoff]

@reuk After some further investigations, the above code works as expected on Windows.

As mentioned in my previous message, if I pass the plugin editor or the webview as the sourceComponent argument to performExternalDragDropOfFiles(), the drag works. It’s when I release the mouse (into the plugin UI or not) that the assert in getNSViewForDragEvent() is triggered.

Here is the context of the call to getNSViewForDragEvent() that triggers the assert:

I can provide a minimal project to reproduce the issue if you want.

[asimilon]

I saw this same assert, but I assumed it can be ignored, because I interpreted it as saying the call to DragAndDropContainer::performExternalDragDropOfFiles needs to have been triggered from a mouseDown or mouseDrag, but of course that event never even happened from JUCE perspective since it happened in the WebView and was proxied back to C++ to initiate the file drag and drop.

[blegoff]

If I comment the assert, nothing happens when I drop the file in the Finder for example. So it’s actually helpful.

The issue comes from Desktop::getInstance().getDraggingMouseSource (0)which returns a nullptr, because we didn’t actually performed a drag before.

[asimilon]

Oh right, sorry my memory is hazy about this, I thought it worked despite the assert.

juce::StringArray files;
// add some files to the array you want to drag out
juce::DragAndDropContainer::performExternalDragDropOfFiles(files, false, &viewContainer);

This is using CHOC instead of JUCE 8, viewContainer is the CHOC WebView:

#if JUCE_MAC
        juce::NSViewComponent viewContainer;
#elif JUCE_WINDOWS
        juce::HWNDComponent viewContainer;
#endif

haven’t got around to trying webview on JUCE 8 yet, but I’m guessing it will be possible to get ahold of the NSViewComponent easily enough.

This works, but the code I was working on using this was mostly being tested in Bitwig without any debugger attached, and on top of that the vast majority of the time not even a Debug build, so I would never see the assert, but it works, so I just glossed over it as a “problem”.

[otristan]

I’m not very proficient with web stuff, but I’m wondering why not use native HTML drag n drop ?

W3Schools.com

[blegoff]

I tried your code suggestion and it doesn’t triggers the assert, but the actual file drop in the Finder doesn’t happen.

NSDraggingSourceHelper::draggingSession:endedAtPoint:operation is actually not called at all. Which I guess is expected for the drag operation to actually complete.

[blegoff]

From what I understand, there is no cross-platform way to drag binary files (other than images) out of webviews. In chromium you have the DownloadURL data format, but it’s not supported by webkit (macOS).

[otristan]

I see. I indeed saw an example with an image and though it would work the same for other file format…silly me.

[blegoff]

@reuk @asimilon I’ve put together a minimal project to illustrate the issue:

GitHub - meteaurestudios/juce-webview-experiments: A test playground for JUCE 8 WebView plugins

[asimilon]

@blegoff

I tried your code suggestion and it doesn’t triggers the assert, but the actual file drop in the Finder doesn’t happen.

That’s weird. I am using CHOC WebView and am able to drag a .MID file into either DAW or onto Desktop without issue.

[reuk]

We’ve got a fix in progress for the assertion-on-mouse-up issue. Hopefully this will be released shortly.

I haven’t seen the other issue you reported, where the target application fails to receive the dropped file. Perhaps it’s worth double-checking whether you’re able to drag and drop the same file from the Finder into the target application.

There was a related issue recently that seemed to be caused by deleting and recreating the dropped file. I don’t see anything like that in your demo project, but perhaps this is something to keep in mind.

[blegoff]

Thanks @reuk. Is the fix you mention tailored for my specific issue, or is it something unrelated that might fix it ?

I haven’t seen the other issue you reported, where the target application fails to receive the dropped file.

It’s when using the code suggestion in @asimilon’s second message (having a static juce::NSViewComponent viewContainer that is passed as the sourceComponent in performExternalDragDropOfFiles).

Also I’ve tested using the PluginEditor::mouseDrag() event to trigger performExternalDragDropOfFiles (when not displaying the webview in the whole window) and the drag and drop happens as expected.

There was a related issue recently that seemed to be caused by deleting and recreating the dropped file. I don’t see anything like that in your demo project, but perhaps this is something to keep in mind.

I’m not doing anything related to file management other than checking the file exists on disk.

[reuk]

@blegoff

Is the fix you mention tailored for my specific issue, or is it something unrelated that might fix it ?

I think it’s unrelated. The change is designed to fix the assertion on releasing the mouse; if the drag is failing for some other reason, then this change won’t help.

This is the commit containing the fix:

Windowing: Store originator component when initiating a mouse drag

Please try out the change and let us know if you’re still seeing problems.

[blegoff]

Hi @reuk thanks for the fix. I can confirm that the assertion is not triggered anymore.

I was then able to find a solution to my issue. The problem lied in the use of the following JS event to trigger the native drag in JUCE, and using a html element with the following attribute: draggable="true".

source.addEventListener("dragstart", (ev) => {
    window.__JUCE__.backend.emitEvent("dragStart",  {});
});

I found out that this JS drag event interfered with the one triggered by JUCE. Since a mouse drag already started (from JS), it would not create the proper macOS mouse event for the file drag (from JUCE). This can be identified by hosting the plugin in the AudioPluginHost, which would log the following when trying a drag:

Cannot start a new drag. A previous drag has not finished.

A workaround for this is to use the onmousedown JS event instead.

Here is the associated commit for my example repository:

fix file dragging issue using mouse down event on div instead of JS drag event

[asimilon]

Ah, this explains why it was working fine for me but you had issues. I probably ran into the same problem but forgot about it, as in my solution I use mousedown/mouseup handlers to just keep track of the mouse button status for the draggable element, and then I use the mouseleave event to simulate requiring an actual drag of the mouse, only calling into C++ when the mouse is down and mouse left the element.