The new WebBrowserComponent additions in Juce 8 look really compelling, and I’m seriously considering doing my whole UI over in JS.
Has any thought been given to how we can link ValueTrees into the webview? I understand that in some ways the new system moves us away from ValueTrees, but in my case the UI is highly dynamic, with sub-components being created and destroyed all the time, and ValueTree is at the heart of this. It would be really great if I could set it up so that the webview Components (in React or whatever) can respond to changes in my ValueTree. Is this possible?
I’m guessing that no such system exists yet, given how new the framework is. Maybe I can ask a more general question then:
What are the best options for sending data and callbacks from C++ to Javascript?
I can see lots of options for sending data and callbacks from Javascript to C++, like withNativeFunction and withEventListener. But I don’t see many options for doing it the other way around. Like I want to have a juce::var somewhere which calls a callback in Javascript any time it is changed. There seem to be built-in ways of doing this with sliders and checkboxes, but what about with a raw value?
You’re right that there is no built-in mechanism for synchronising ValueTrees with the JS context inside the WebBrowserComponent.
Your primary tools for sending data to the WebView are WebBrowserComponent::emitEventIfBrowserIsVisible and WebBrowserComponent::evaluateJavascript.
Out of the two emitEventIfBrowserIsVisible is the higher-level tool, and probably the easier to use, under the hood it is calling evaluateJavascript. This would allow you to emit any var under a given event name, and you can subscribe to these events with window.__JUCE__.backend.addEventListener in Javascript. This example is a bit more detailed in the function’s documentation.
Another pattern, which you will find in the WebViewPluginDemo, is to just send “observer notifications” using this function. Your Javascript event listener can then call a native function, or if it wants to receive a lot of data a more efficient way is to fetch data from a special URL address that you serve from your ResourceProvider in C++.
Hey @attila , can you elaborate why going through the resource provider address is more efficient than for example just calling through a native function and returning the data on complete? I saw that that’s what you’re doing for the spectrum data and I’m wondering what the benefit is over a native function call for this.
//Edit: after digging and testing a bit, could it be that regular native function calls always go through a toString() stage at some point and going through the Resource Provider doesn’t?
I managed to send raw pixel data for an image through the resource provider and it arrives as raw uint8 array on the JS end, going through the native functions always yields objects of one sort or another with being converted toString() in JUCE at some stage. Am I getting this right?
With native functions you can only pass var objects and receive a Javascript Object. The transit will always include a serialisation to and from String. You can bypass this with a ResourceProvider.
My guess is that the native function / event route will generally result in nicer, easier to write code, so it can be a good first approach. I would advise against trying to prematurely optimise and shove everything in a ResourceProvider. But for binary data, like image files, audio samples The ResourceProvider seems like to more natural fit. Also, if you find a performance issue caused by the overuse of native functions, you can think about how to fit it into the ResourceProvider approach.
Yeah I’ve already implemented both, it’s pretty straight forward. One drawback of the Resource Provider I had to work around: you can’t pass any more parameters in than a sub-path when you’re calling it from the JS frontend. Is there a possibility to have the Resource provider call the getResource function not with a string, but an URL object instead, which we could attach POST data and/or parameters to on the front end?
Either way, I realized (I think?) that neither approach is suitable for what I’m trying to do.
I’m trying to push(/poll) fully rendered frames for complex UI components to the frontend at framerate and I think it’s not possible with either solution due to the constraints of http calls (please correct my dangerous half-knowledge if I’m wrong).
So now, I’m 90% done implementing a WebSocket with juce::StreamingSocket to see if that works better, which all my research points to that it should…
//Edit: yes, that did it
This is a juce::Image rendered on a background thread using juce::Graphics (just counting the frames) at 30fps in JUCE, 800x600 pixels and then transmitted via websocket as raw pixel data to the browser frontend, directly displayed on a canvas in a Vue app. Can’t feel a thing handling buttons/knobs on the same, single-threaded frontend.
//Edit 2: now - transmitting an actually moving image - i can unfortunately see that it’s super janky albeit the transfer IS really fast… I’m not sure yet that this can be done well, I’ll report findings when/if I should get them… don’t follow my lead just yet if you read this and get excited by that gif up there, writing a websocket server suuuuuucks
Well, expanding on my last edit, I got to findings a bit faster than expected. It’s definitely doable:
The gif makes it look a bit worse than it actually is. It’s not perfect yet, but I think there is a manageable way using a web socket. I figure I should toy around with compression a bit…
Sorry for derailing this thread a bit from its title, but i think it touches on the same issue of large-data transmission between JUCE and the webapp that displays in the WebBrowserComponent I can report that websockets are definitely a super-fast way, if the Resource Provider is not enough for some reason, but they’re also quite hard to implement. I might share something, if I make the final jump to JUCE 8 and it’s worth investing more time into the code.
@benediktadams did you end up adopting this approach? I’m about to embark on the same journey. I’d love to know if you’d be open to sharing what you’ve written so far!
I never got to wrap this up unfortunately… I just looked over the experiments I did, and while the entire rendering pipeline is kinda complex, convoluted and tied into my systems, I did find a part that is integral and that I can share easily enough.
My entire approach is centered around using a Web Socket for blazing fast data transmission between the JS frontend and the JUCE backend. To facilitate this I wrote a Web Socket protocol integration using the JUCE StreamingSocket class. So the idea is:
Host a Web Socket Server in the JUCE backend
Use existing JUCE Web mechanics to communicate web socket address and port to the front end
Use any frontend web socket library to hook up to the web socket.
At this stage it’s deciding on your data format and how do read/write it on both sides of the thing.
In my experiments I just used a juce::Thread, created a juce::Graphics object that renders to an juce::Image and then I transmitted the raw pixel data via web socket, which I then painted onto a OpenGL canvas in the frontend.
I pulled out my web socket code and stuffed it into a module for you, quick and dirty… I haven’t even tested it, but you can at least get an idea. Feel free to fork and fix up what needs fixing up (I also slapped an MIT license on it so you can safely use what you need/can)
It does integrate a lot of valuable stuff like web socket handshake / heartbeat or accept key computation according to RFC 6455. So instantiating an instance of this class should already get you a running WebSocket, that you can hook up to from your frontend.
//Edit: just in case that’s not clear, the repo has a submodule (TinySHA1), so clone using the --recurse-submodules flag or init submodules after cloning.
![[video-to-gif output image]](https://us1.discourse-cdn.com/flex026/uploads/juce/original/3X/0/8/0809fd957af2200c0eb9b287bec0bf4ecd83d61d.gif)
I can report that websockets are definitely a super-fast way, if the Resource Provider is not enough for some reason, but they’re also quite hard to implement. I might share something, if I make the final jump to JUCE 8 and it’s worth investing more time into the code.
(I also slapped an MIT license on it so you can safely use what you need/can)