My team and I have been experimenting with Direct2D as we set ourselves up to move from JUCE 7 to JUCE 8, but we’ve encountered an issue that seems to be dependent on which renderer is being used.
What we’ve noticed is that though Direct2D seems to perform very well in terms of raw frame rate, having 2 or more instances of our plugin open at the same time seems to completely gum up the mouse interaction. After opening 2 instances, the mouse interaction completely freezes up and I can’t interact with the plugins nor even the DAW itself (Ableton Live) using the mouse without a massive (10-20 second) delay in the interaction.
I’m currently trying to track down the source of this issue. I’ve built some of the JUCE example plugins and attempted to reproduce the behavior with them, but they don’t seem to cause the same issue. I’m working on creating a minimal reproducible example right now. Debugging/profiling hasn’t revealed anything of note to me yet. I’ll post a minimal reproducible example if I can generate one, but I wanted to post this now in case anyone is experiencing similar issues or can give me some guidance based on the information I currently have.
One important side note: using OpenGL to render the entire app also appears to cause this freezing of mouse interaction, but the software renderer doesn’t have this problem. This of course leads me to believe that this is an issue with misusing/abusing GPU resources, but so far I can’t see where that’s a big problem in our code.
Which JUCE version are you using for testing? 8.0.3 included several fixes for Direct2D, so I strongly recommend updating if you’re not already using that version.
Have you tested on multiple machines, and do you see the same behaviour on all machines if so? It would be interesting to test the both Windows 10 and 11, and a variety of graphics cards.
Is it only the mouse that freezes, or does the entire computer freeze? I wonder whether it’s possible to hit “pause” in the debugger at the point of the freeze and to inspect the stack trace to see whether there’s some lock contention, or a very expensive function, or something else going on.
We’ve seen similar issues across the windows machines our teams use. My machine (RTX 3090 Ti, Radeon 5800X) is where I’ve managed to reproduce the worst of the freezing, but we’ve also seen similar results on smaller Windows laptops with integrated graphics.
Regarding the exact freezing behavior: the plugin window and the DAW window freeze (but don’t show as “Not Responding”), and other OS windows behave normally.
The branches I’m testing on are using 8.0.2, so I’ll bump the version on those and report back once I test that.
However, as I’ve dealt with the issue today, it’s become clear that it’s probably due (at least in large part) to improper allocation of resources in our code. The biggest offenders were instantiating up to a dozen ColourGradients within the paint routines instead of caching them as is suggested in the docs. I’ve since fixed that code to cache the ColourGradients, but that’s only helped the problem a little bit. I’ve still got two questions that are bugging me:
Why does the frame rate remain high while what appears to freeze is mouse interaction? I don’t know enough about JUCE’s internal implementation or Direct2D to make any kind of educated guess about that.
Which resources are expensive to instantiate from a graphics perspective? The docs of course mention that Image and ColourGradient are obvious examples, but I’m unclear on whether often-used classes like Path also incur expensive creation calls to Direct2D on instantiation. If there’s some more documentation regarding this that I’ve missed I’d definitely appreciate being pointed towards it!
Thanks for your response, I’ll keep posting as I get more information.
Just a quick update on this: I’ve narrowed the issue down (slightly) to some of the code we have that connects our audio parameters to the UI. We have a ton of custom code around this so I doubt it’s an issue with JUCE’s parameter logic. I’m still being thrown for a loop as to how which renderer is being used affects this, but it’s becoming clearer that it’s an inefficiency on our end. If it ends up being some kind of issue with JUCE as well, I’ll be sure to post that up here or in a new thread.
I also couldn’t guess at the reason for this. Perhaps you could try enabling Direct2D debug output as described in this post and see whether this provides any helpful information.
The best way to answer this is with a timing profiler. That way, you can determine exactly how much time is spent creating each resource.
It’d be good to check that you’re not attempting to do any drawing directly in parameter change callbacks. If you have very rapid parameter changes, you could end up doing a lot of unnecessary drawing unless you throttle the updates down to the screen refresh rate. Additionally, painting won’t be realtime-safe, so painting directly in parameter callbacks may cause audio glitches.
I certainly don’t think we’re doing any painting in the parameter callbacks, just some calls to repaint() to mark the components as dirty. I’ll look into it to make sure, but the evidence is pointing more and more towards irresponsible usage of resources in paint routines. Thanks for the tip about the D2D debug mode, I’ll check that out as well!
Update: after making a minimal reproducible example, it’s become clear that this seems to be an issue with JUCE + D2D that differs from the software renderer. You can see my example code in this repository. I’ve tried a couple of permutations, and the main offender seems to be using a lot of ColourGradients.
Obviously, this example is a massive overuse of JUCE’s graphics. Rendering dozens of random gradients every frame is clearly a huge no-no. However, the part that still concerns me is the fact that only on D2D, the entire DAW will freeze up along with the program window well before the framerate dips that hard on the plugin UI itself.
(Obviously this is a call on our side to refactor aggressively, as this problem doesn’t occur unless the paint routines are way heavier than they should be. However, I’ve also bumped into an issue regarding effective optimization of ColourGradient, which I’ve outlined here)
If this is simply a symptom of worse performance, and it’s normal that a clogged render queue will block the message thread, then I’d consider my current issue resolved and pivot to viewing this as an optimization problem. Just wanted to make sure that this was on the JUCE team’s radar in case the mouse interaction lag isn’t expected.
P.S. I’ve tried your D2D debug tip, but nothing seems to come out in the logs.
Just fiddled with my demo project a bit more and it became clear that the mouse interaction lag is directly proportional to render performance, regardless of render engine. I’d consider this issue resolved, but still have concerns outlined in my other forum post.
Thanks Matt, great tip! I had no idea about process explorer. It reports that the DAW is only consuming about 100MB of VRAM, but there’s a ton of other great metrics in here that I can use to zero in on the problem.
Just FYI: I was also wondering about memory consumption for our products and a hobby project, and I’ve found that the debugger in MSVC adds ~100 MB to the total in their profiler.
If you run your app just by double-clicking the icon and checking the task manager, it will suddenly be ~100 MB less.
