Another software renderer?

Revisiting the topic of Skia. I recently learned that a certain synth plugin I use uses Skia for it’s rendering, software only. I was convinced they were using hardware accelerated graphics after using it.
From this post Is the JUCE team still here? - #22 by t0m it seems that the JUCE team didn’t think Skia made much difference.

I still don’t understand why JUCE’s rendering is so poor in terms of animation performance, and why a non-JUCE plugin using Skia software rendering looks as smooth as a video game.

2 Likes

I’ve tried using Skia in a JUCE app and it’s an absolute nightmare. It was a while ago now so I forget the details but I remember spending several hours just running installation scripts and googling error messages. There’s no way a library like that could be integrated with JUCE in a clean way - even a third party wrapper of Skia in a JUCE module would be difficult.

As for why JUCE’s performance is ‘bad’, I don’t think it is. I think it’s just that the way to get good performance out of JUCE isn’t always intuitive and you have to be very conscious about what you’re rendering and when. Of course there’s only a limit to how well software rendered graphics can perform, so I would love to see a more modern cross-platform alternative to OpenGL in JUCE, especially for things like shadows and blur effects that are always going to perform badly with a software renderer.

My biggest advice would be to use a profiler and actually see what’s taking the majority of the time when you’re rendering your UI. You’ll often be surprised how easy it can be to get a huge performance boost from very trivial things. For example, with a quick test I just managed to draw a simple label 16x faster by buffering it to an image, rather than letting it be painted as normal - which is why I always buffer my labels to an image and never draw text anywhere outside a label.

8 Likes

just my two cents

I think the the main problem with smooth animations is the “cooperative multitasking”, because all plugins sharing the same message thread “time-slots”. Also most program-logic happens on the message-thread. I wonder why this aspect is so rarely mentioned.

Even the juce OpenGL renderer needs to lock the message-Thread whenever it wants to draw any component based stuff.

The key for smooth animation is to decouple the message-thread from the graphic output.

For example, you can draw on an juce-Image on a background-Thread (via software-renderer) and use this image on the renderOpenGL()-callback (which of course uses memory-bandwidth)

13 Likes

Does Juce still split the graphics into pixel wide slithers? I’m sure I’ve heard that somewhere on the forum.

Yep. To be specific, you’re referring to the EdgeTable which basically turns out to be a scan-line/rasterising rendering method. I’m sure it made sense at the time of its creation, but in the modern tech world it’s a terribly inefficient approach. Using the main thread to load all of the information as pixels, composing the contents into a giant image, bypasses the whole point of a GPU pipeline.

In the same vein as Chkn mentioned;
From my experience, any decent rendering engine today is multi-threaded: hold the graphical objects on a dedicated UI thread - well away from the main thread - where the user queues up any changes to the UI objects (the user side or JUCE’s side should/could hold handles to all of these elements). This approach allows you to control the update pace of the UI thread and getting nicely synchronised frames (aka well-timed animations), and with the right strategies in place - like culling - would give excellent performance. This particular approach is what I used when porting video streaming software from high-end platforms (eg: PC, game consoles) to some of the most craptacular devices imaginable (eg: mobile devices, set-top boxes), all while maintaining vsync with some of the most heavily animated UIs I’ve ever dealt with.

This is all akin to Skia from what I’m reading (I haven’t used it… yet?). They do state how they support the gamut of low level GPU pipelines and allow plenty of control when drawing.

For how the JUCE devs tested Skia, I can’t say. But if the approach to testing Skia was to just draw everything onto an image, like how JUCE normally does things, then it wouldn’t show any difference… (and would miss the point of using Skia)

JUCE forum bias is my best guess - with audio being the focus, everything else tends to fall to the wayside (priorities…).

7 Likes

I also think it is time to improve the software renderer. It worked well ten years ago but in my opinion, problems started especially with the high-resolution screens on macOS.

It’s hard to get a fluent animation with the current software renderer in a plugin and it uses too many resources when you try to do this. I often have to change the requirements because things are not possible.

I would prefer if we could improve/rewrite the JUCE renderer over an external library like Skia that contains a huge amount of additional code and is done for much more complex use cases we may don’t need. I also fear the increased compile time.

On the other side, the time has changed and maybe it’s not possible anymore to make a fast renderer ourselves that works on all platforms. Same for the SVG support in JUCE.

3 Likes

Aside from the difficulties in actually getting it to work well in JUCE, my main concern with Skia would be that it’d soon end up on https://killedbygoogle.com/

1 Like

I’d love an update on the renderer front. This has been requested for half a decade now. There was a time a few years back where it seemed like the devs were looking into options. Anything to update us all on?

Skia is massive though, and the basis of all Chrome products among many others on many platforms.

&

Skia Graphics Engine - Wikipedia.

I don’t think it’s going anywhere soon.

1 Like

People probably said the same about Angular, which is near the top of that list.

Angular has ~60k stars on github, Skia has ~6k stars.

1 Like

They, and Apple now both use ANGLE

Which has had a recent Safari and iOS (metal) upgrade to Web GL2 , which is why a lot of shaders now work in JS.

1 Like

IMO the best approach for JUCE would be to implement native graphics APIs (already has direct 2d on windows, just need to add metal support on mac) - and then revamp the graphics context’s for those APIs to make better use of the GPU. Currently, switching to using one of these back-ends gives very little performance gain over the software renderer since the majority of the work is still done on the CPU.

Otherwise, JUCE’s graphics API is just a layer on-top of some other graphics API that’s also just a layer on-top of native APIs. Why not cut out the middle-man? If JUCE were to use something like Skia, I’d just end up wanting to use Skia directly to get the most out of it, in which case what’s the point in JUCE implementing it in the first place?

4 Likes
  • It’s uses a common way of rendering 2D graphics quickly.
  • Secures Juce’s immediate future of hardware interaction in an increasing multi-tasking world.
  • It’s a better alternative to writing everything from scratch on every platform. And spend years testing and adjusting things to work, as they all change ( I don’t think there’s enough staff around to do that)
  • Open source

I don’t know. It’s not like me to be negative, but I feel like I’m wasting my time as I don’t think anyone here wants to tackle anything on the renderer side. Juce seems to be fundamentally Mac devs on the whole, and the fact the there is no Metal or Vulkan with the current Juce API is quite demoralising.
‘Parawaves’ attempt is amazing though…

2 Likes

I don’t disagree with any of those benefits, but simply laying JUCE’s current graphics API on-top of some other graphics API isn’t going to give the benefits people are expecting - as discussed in this thread, using Skia as a backend for JUCE in its current form doesn’t give much of a performance boost. Hence why even OpenGL isn’t really a viable option - you can very easily change your JUCE app to use hardware accelerated graphics in just a line or two of code, but the performance gains are next-to-none as JUCE doesn’t properly utilise those APIs.

Which is why I think the better approach would for JUCE to simply use the native APIs, but change how it implements them to get the most out of its performance. That would make JUCE a viable alternative to these other 2d rendering APIs, and give it a wider use outside of just audio apps.

Essentially, make the JUCE graphics API hardware-first. Add Metal for mac, possibly replace Direct2D with Direct3D for windows, keep OpenGL for linux (?). Then add a software renderer as a fallback to support older machines.

1 Like

Trying to squeeze Skia into the current Juce is a waste of time, I agree.

I worry that Juce users think hardware rendering is slow, but it’s just the way Juce presents the data.

I do find Juce’s OpenGL invaluable though. Especially with line drawing and scaling massive bitmaps.

gah, I just want them to start something. :grin:

2 Likes

Crazy Idea:
Juce should fork flutter, remove all the DART stuff and rebuild Juce on top of flutter’s C++ engine (which uses Skia).

https://raw.githubusercontent.com/flutter/engine/master/docs/flutter_overview.svg?sanitize=true

Less crazy idea:
Juce should create a more limited (but GPU based) 2D graphic api using @tayholliday1 vger library.

4 Likes

I’ve recently been considering combining Flutter and JUCE for a client desktop app for improved UI performance… but I’m not (yet) sure it’s worth the extra complexity vs pure JUCE.

1 Like

It’s hard to see where we can made a dramatic difference to rendering performance without a very substantial rewrite of JUCE’s internals.

We implemented a Skia backend for macOS, to see how it compared to using CoreGraphics. This wasn’t just drawing everything into an image, this was translating all juce::drawPath and friends into skia::drawPath and similar where Skia was configured to use its Metal backend. Unfortunately this didn’t provide a consistent speedup over CoreGraphics, and also added approx 40 mins of compile time on a fairly recent Macbook. In hindsight the lack of a rendering speedup is not that surprising. Where CoreGraphics is rendering to the screen (the normal JUCE rendering mode) Apple’s engineers will have taken effectively the same approach as Skia - opaque drawPath commands will have been translated into something the GPU understands as efficiently as possible.

So “JUCE should have its own GPU renderer” is not the whole of the problem. Our testing with Skia’s GPU renderer has shown that with conventional layered 2D drawing we are unlikely to do better than CoreGraphics (or Direct2D on Windows). There are some places where we could gain a bit if we restrict transforms to scales and translations and remove the ability to clip to paths, but it’s not clear how much faster that will make things. Qt has something like this restricted mode to render a bit faster. To make a more substantial difference we will need to change how graphics state is cached on the GPU and potentially just communicate updates, in a similar way to that suggested in this thread, or move to a different model like React’s virtual DOM or Flutter’s caching.

We’re investigating alternative approaches, and we’re also keeping our eye on WebGPU and efforts like Google’s Dawn that would allow a cross-platform implementation of shaders.

18 Likes

Thanks for an update. I’m intrigued - When you say ’ translating all juce::drawPath and friends into skia::drawPath" - doess that mean RAW painting or does it still use anything Juce on top, like edge-lists?

Google’s DAWN and ANGLE projects look promising, but so does @parawave 's Vulkan work.

Using Vulkan is a quick compile, and works just like OpenGL did. It handles multi-threaded environments and reduces the need to alter Juce renderer fundamentally. Plus with Vulkan or Molten in Juce would be a new starting block for gradual future changes.

I’m not sure what RAW painting means. No edge tables are involved in the CoreGraphics renderer and no edge tables were involved in the Skia renderer either.

2 Likes