Trying to fix JUCE_COREGRAPHICS_RENDER_WITH_MULTIPLE_PAINT_CALLS

Jules originally added the JUCE_COREGRAPHICS_RENDER_WITH_MULTIPLE_PAINT_CALLS option to work-around CoreGraphics’ inability to tell what components are not in the clip region, and for a while it worked.

After more changes from Apple’s side it stopped working, with some confusing interim states where using certain build settings did work, but as for now on my 10.14.6 system no build settings I know of works and to add injury to insult it doesn’t work in a nasty way!

Nasty way of not working: If you rely on JUCE_ENABLE_REPAINT_DEBUGGING to catch unnecessary repaints - it will mislead you! You can see for yourself by placing a breakpoint on the paint method of something that seemingly isn’t repainted but which lies between animating components (level meters etc) and you’ll see that the paint method is being called even though JUCE_ENABLE_REPAINT_DEBUGGING tells you otherwise!

Here’s a new work-around that does seem to fix everything:

What Apple broke is the [NSView getRectsBeingDrawn: count:] method which we can’t rely on anymore to return the right result, and the work-around is to maintain our own RectangleList of dirty rects like for example the OpenGL renderer does.

If anyone has this problem I’d love to hear if this works well for you. Cheers and stay safe! Yair

9 Likes

Nice job. I’ll try this next week.

1 Like

We will merge this into our branch, because we have Mac customers who experience bad performance because we have quite a lot of text between timer-updated graphics and it paints roughly 80% of the UI because of that problem. It should paint maybe 10%.

2 Likes

Maybe we should join our forks into one (we auto-merge with the official “develop” once per hour) so we can both benefit from each others work on JUCE without having to cherry-pick each others changes?

Sure, perhaps we should add READMEs in the branches describing the diffs from main JUCE. It might take some time to gather for our branch, where the major diffs are Offline AudioSuite and ARA plugin-side stuff

1 Like

An update on my fix: it has issues with moving components.

Unfortunately there’s no guarantee that the [NSView drawRect:] call should draw all of the rects orders via [NSView setNeedsDisplayInRect:], so assuming we know which rects need updating in the call is not always correct. I’m stuck :frowning:

I’m currently testing your branch. I’m seeing issues where showing components causes the components underneath to bleed through. Is this what you are seeing:

Could the ComponentPeer keep a Image that buffers the entire component. When the OS asks to paint, just a section of the buffer is copied out? And repaint causes the image to get updated. Would this be any different than calling setBufferedToImage(true) on your topmost component?

Yeah I got similar issues but which only triggered by moving components.

I think that when [NSView drawRect:] is called, sometimes it refreshes only a subset of the prior [NSView setNeedsDisplayInRect:] calls (with the remaining coming in a subsequent drawRect), so my assumptions that all dirty rect are supposed to be drawn was wrong.

Your setBufferedToImage work-around sounds interesting. Haven’t tried it, please report if you do :slight_smile:

I think that I have a good understanding of the issue and a possible solution. It will probably take a while until I’ll get to it (for now using OpenGL is a good work-around for us) but if someone wants to implement it here are the details:

  • The faulty fix assumed that all the dirty rects should be drawn in drawRect, but actually only some of them are supposed to be drawn
  • Drawing a dirty rect that the drawRect call wasn’t intended for has no effect! Only the rects it was called for are copied to the buffers in the video card
  • In a later call that is intended for that drawRect, we’ve already removed it from the dirty rects collection and not draw it - so we get old stuff drawn.

The work-around I came with (and haven’t implemented) is to play detective on which rects that need to be drawn according to the rect received in drawRect -

  • Collect the dirty rects in order in a vector (not a RectangleList which potentially merges or reorders them)
  • Let’s assume the first drawRect call
    • Iterate over the dirty rects and construct a bounding box of them until it matches drawRect's rectangle. These rects are certainly rects that this call is intended for - draw them and remove them from the dirty rects vector!
    • Continue iterating over the dirty rects. If a rect is contained in the big rectangle then we don’t know if it can be removed from the dirty rects vector, so we don’t and we draw it even though we may later draw it again!
    • If in the iteration we find a rect that sticks out of the big rectangle then we know that from this point the dirty rects are not part of the drawRect call
  • For subsequent drawRect calls we have potentially spurious rectangles at the beginning of our dirty rects list, which we did draw in previous calls but did not know that it is safe to remove them from the list. So we as we do the first iteration described above, if we haven’t yet constructed a bounding box which matches the drawRect rectangle and we come upon a rect that sticks out it, that means that it must be such a spurious rect, so we remove all rects until it in the dirty rects list and start the process again

It’s a bit complicated and has an assumption on how drawRect unifies smaller rects to a big one (that it does it for consecutive rects), and it’s not clear if the overhead of sometimes drawing rects twice will not be worth it. I guess we’ll find out some time :slight_smile:

Is this still really useful with JUCE_COREGRAPHICS_DRAW_ASYNC enabled ?

Yes, it is. What triggered me to look into this was that in profiling my program (which was sluggish) it spent way too much cpu time in drawing buttons, even though no buttons needed drawing!

While JUCE_COREGRAPHICS_DRAW_ASYNC may affect the performance of the Core Graphics calls, it will not help avoid traversing unnecessary components in JUCE’s component drawing code.

Just to be clear, are you saying, you still had major issues after JUCE_COREGRAPHICS_DRAW_ASYNC was enabled? Or did you not even try the ASYNC option?

Just to verify that I’m not making baseless claims, I’ll try that route too and report my findings

So I ran my program, Release build, on my machine (MacBook Pro 13-inch, 2016 Four Thunderbolt 3 Ports running macOS 10.14.6) with several different options, and wrote down the CPU usage.

The CPU usage results are simply based on looking in Activity Monitor, which sometimes fluctuates over a large area and the figures are some approximate middle value. The machine ran Xcode and a browser at the same time so it’s not a “clean benchmark”.

Variant CPU Usage
No special flags 25%
Using OpenGL 9% [0]
Many setBufferedToImage calls [1] 10%
USE_COREGRAPHICS_RENDERING=0 65%
USE_COREGRAPHICS_RENDERING=0 with dirty rects fix [2] 50%
JUCE_COREGRAPHICS_DRAW_ASYNC=1 23% [0]
JUCE_COREGRAPHICS_RENDER_WITH_MULTIPLE_PAINT_CALLS=1 8% [3]
JUCE_COREGRAPHICS_DRAW_ASYNC=1 with JUCE_COREGRAPHICS_RENDER_WITH_MULTIPLE_PAINT_CALLS=1 12% [4]
JUCE_COREGRAPHICS_DRAW_ASYNC=1 with setBufferedToImage calls 14% [4]

YMMV! The tested program is pretty simple, has some animating stuff around and lots of non-animating stuff in between, including buttons with text. Some options performing better than others doesn’t necessarily means that would be the case for all programs.

  • [0] When using OpenGL or JUCE_COREGRAPHICS_DRAW_ASYNC, activity is off-loaded from main-thread, so we should expect better overall responsiveness than the raw cpu usage numbers indicate
  • [1] setBufferedToImage (true) for all Buttons and Labels in the UI and some more mostly static components
  • [2] Part of underlying problem which JUCE_COREGRAPHICS_RENDER_WITH_MULTIPLE_PAINT_CALLS tries to address is an NSView problem rather that a Core Graphics problem. Its fix also affects performance also when using the software renderer
  • [3] This is using SR’s branch with the partially fixed, sporadically broken JUCE_COREGRAPHICS_RENDER_WITH_MULTIPLE_PAINT_CALLS implementation
  • [4] This surprised me! JUCE_COREGRAPHICS_DRAW_ASYNC=1 has a little bit of CPU usage contribution when used alone but not when used in conjunction with other measures. It might still be worth it as it off-loads work from the main thread
1 Like

Are these numbers just based on Activity Monitor? If so, this probably isn’t a good test of UI responsiveness which is probably the issue you’re trying to solve.

When using the JUCE_COREGRAPHICS_DRAW_ASYNC=1 flag, although the juce::paint methods are still called, the drawing actually happens on a background thread so you’ll probably see a much more responsive UI. (This was our experience and we have a very dynamic and animated UI). Using a PerformanceCounter inside the ComponentPeer paint method is probably a better indicator of responsiveness.

Whenever I’ve done UI profiling most of the time is usually spent in the CoreGraphics calls (our juce layer is usually inexpensive or can be optimised pretty easily e.g. by caching GlyphArrangements) so reducing these (by using the async drawing) has been the biggest win for us.

Having said that if you can get the multiple paint calls to work in conjunction with async drawing that would probably be the best approach.

Added that test to the table. Surprisingly it didn’t contribute more than JUCE_COREGRAPHICS_RENDER_WITH_MULTIPLE_PAINT_CALLS on its own. Might possibly be worth further investigation.

Is JUCE_COREGRAPHICS_RENDER_WITH_MULTIPLE_PAINT_CALLS in the current dev-branch working? imho no

Thanks @yairadix-2 for the patch, but i suspect it to create some sort of heavy flashing like a stroboscope between often repainted areas (catalina, xcode 12 beta 5, macbook pro 2018)

Yeah my attempted fix ended up not working. We’ve ended up keeping using OpenGL instead. I’ll revert my changes so folks would be able to use JUCE_COREGRAPHICS_RENDER_WITH_MULTIPLE_PAINT_CALLS in our branch. Thanks