Now that we have good support for web views in JUCE, and we also have some nice graphics enhancements in JUCE 8, I’ve come back to a question that I’ve had hanging for a while now:
What are the pros and cons of creating a plugin UI with web views, and in what situations might one be preferable over the other?
I have some thoughts on this myself but I’d like to open it up to the community.
As one of the guys really excited about the new web view possibilities, I’ve encountered some immediate issues where the current state of web view UIs are lagging behind native UIs:
In general, I’m still optimistic for a future where the majority of the UI can be web based, just because it’s vastly easier to work on, with a much faster development cycle. But be aware of browser limitations, because they bite you whenever you go to the weird parts of UIs that were never supposed to look and feel native.
Pros
Cons
It seems that, for classic synth and effects UIs with fine-control knobs and frequency analysers, JUCE graphics may still be the way to go.
However for plugins or apps that are more based on multi-level menus and dialogs with less interactive graphics, the web view route seems quite attractive.
I wonder if leveraging WebGL might solve some of the issues you’ve come across @TobbenTM ?
WebGL will definitely help with any sort of frequency/spectrum analyzers, however, it still needs to be built, and it’s often some of the most complex parts of plugins.
I think that until now, there just hasn’t been much of a market for these kinds of implementations for web stuff, at least nothing this high performance, so it feels like we’re treading new waters building this.
Not to mention the data pipeline from the Juce backend also needs to be smooth and consistent.
As for the particular issues I’ve had with knobs and controls, my current implementation relies on capturing the pointer while adjusting the knob, and this works for all major browsers, but not in the Juce web view.
Still investigating here, so maybe I’ll have something functioning and open source in a little bit, I’m sure it’ll be useful for more of you.
At least, the Juce8 webview2 feature helps to get quick start on communications between Web technologies and Juce. But graphically speaking, I’m not sure it really helps to gain development time (I even don’t speak about performances).
Regarding development time, I think it really depends on the type of UI.
For a UI that can leverage existing web components with a little styling adjustment I think the development time would be drastically reduced, e.g. for UIs that consist of lists, comboboxes, dialogs, tables etc. Also for adjusting layouts. Removing the compile step and having the UI update as soon as the file is saved is a massive developer momentum boost.
For custom graphics I’m not sure, although with vector graphics at least, a lot can be done with SVG libraries.
Hot reloading is great feature for sure
is it possible to render a WebGL pixel shader?
Having worked at a custom development shop for decades that did mostly web work, I expect a lot of dashed hopes in the JUCE community. You can get something quick and dirty working quickly and dirtily using either approach. Making a polished professional system has difficulty that’s inherent, and you can’t (by definition) get around that. Also, most of the hard research/work that’s been done to improve web development is focused on handling text and static images. not exactly on point here.
Happy to be proven wrong, but my projects would all have benefitted more from things like improved mobile OS support.
edit: for kicks, do some searches for things like “why do my SVGs display differently in different browsers?” JUCE’s SVG renderer may be idiosyncratic, but it’s idiosyncratic in a way that’s uniform across platforms.
would love to know some specifics about differences between Webkit and Blink when it comes to building UIs. I’ve been searching around, including using ‘why do my SVGs display differently in different browsers?’ and not found that much to get excited about yet.
This is about the best i’ve seen so far. Any other actual resources would be great.
this is a theme that comes up again and again. i’m with you on this.
i might be really missing something, but I just don’t get the webview stuff at all - esp when it relies on something that can be changed and updated causing breaking changes. why would anyone want to subject themselves to that, as well as having to target multiple different web techs?
I wonder if the new webview let us to access file system (read/write) on iOS devices. Anyone knows ?
Apart from that I think it is a great feature…I could have the entire UI on a cloud server and end users can update the plugin UI without any reinstall (or new App store publishing).
Only the c++ engine update would need a redeploy/reinstall
When I do web layout, I mainly use flexbox or grid layout to make a flexible layout that lets the user adjust the windows, tables or dialog layouts to suit their needs. That means the layout should look good and be functional what ever aspect ratio or size etc they prefer.
When coding UI with Juce, I do the same, I use flexbox or grid layout. Therein I put listboxes, dialogs, text areas, knobs etc. Same type of layout job as if doing html. And same result (more or less). I can’t see why doing this in html should take much less time, if any, than doing it with Juce.
And regarding the “massive developer momentum boost” of not having to recompile, I haven’t a clue what you mean. Pressing Alt-F10 to apply code changes in VS, literary takes no more time than doing a file save. Okay if I make some major changes that neccesitates a recompile and linking of the file and a restart of the debugger, that takes about 1 second. Or maybe 2, but I can live with that.
I started talking about something related to all that in this thread over here
There are quite some things that are just missing from most web UI frameworks, that we need in audio software. As @TobbenTM mentioned, knobs are a great example… just never needed on the web, so they’re basically non-existent when it comes to certain, needed functionalities.
I’m super excited about the possibilities, but at the same time we’ve spent a lot of time and money over the years building some complex UI components that render very performant in JUCE, and I cannot see us recreating these with the tools available in web frameworks.
So it started investigating, if I could just render certain elements through juce and display them in the web view, et voila:
It’s of course not perfect yet, a bit janky (the gif actually makes it worse than it is), but this is rendered in JUCE just using juce::Image and juce::Graphics to call into our paint routines, and then displayed in a Vue app, without affecting performance of the Vue app.
To accomplish this I implemented a Websocket that just streams raw pixel data on a dedicated thread, after rendering on another dedicated thread, all managed lock-free (Cannot recommend…) and then displaying it on a webgl2 canvas (can also not really recommend if you don’t know GL).
As I stated on the other thread, I should toy around with compression, because…
But at this stage I gotta leave it here until I know that I can make the jump to JUCE 8.
I think ultimately there is a way to utilize the best of both worlds with a bit of tinkering, and one can potentially fill out blind spots that web frameworks still have when it comes to audio software UI. I know people like @TobbenTM are working hard to get some native web elements going, that perform well for our special audio software requirements, so hopefully Web UIs become actually feasible very soon.
I think the main argument is iteration time. Even if you’ve optimized your build time, moving an element by 1 pixel takes like 15-30 seconds to rebuild and test. My approach to addressing that concern would’ve been some kind of JSON configurable layout helper. Separate the functionality from the presentation and allow the latter to be hot-loaded from JSON so you can tweak in real-time.
There are other benefits to having a web stack available, but the last thing on the planet I want is audio plug-ins constantly making web requests. Many users don’t want your plug-in touching the internet at all unless they’ve explicitly given it permission in that moment. I agree with them and won’t install “cloud” plug-ins at all. Local is better, but I have a feeling most of these plug-ins are going to feel pretty clunky and are going to consume memory like crazy.
You just explained my plugin-gui-magic 
just that it isn’t JSON but a ValueTree
I was also concerned about memory usage, so far I only have useful info on windows.
I’ve built a plugin with a UI that is 33mb of images.
There appears to be a rough base cost of 100mb for the WebView2 Manager process which hosts all the plugin UI instances.
When loading an instance it can take up to an extra 80mb or so, but then settles down to roughly 35mb, not much more than the size of the actual assets.
I need to run some tests on mac to see how the OS supplied webkit behaves.
What about when you’re working on a very large project that takes 30 seconds or more to recompile, link and restart the debugger? Compared to 1 second with hot-reload.
yes, i use the flexlayout stuff also so i find coding UIs really quick…
