Again: unnecessary Calls to paint() MacOSX

Audio Plugins
1

Here is a simple Demo-Component.

It owns two opaque components. If you repaint() both of them at the same time, the parent-component paint() method will be called too (but it will not redraw, confirmed with random colour-fill).  So if there is nothing repainted, why is it called?

Maybe room for huge performance improvment?

 

class MainContentComponent   : public Component, public Timer

{

public:

    

    MainContentComponent()

    {

        addAndMakeVisible(c1);

        addAndMakeVisible(c2);

        

        setSize (600, 400);

        startTimer(100);

    }

    

    ~MainContentComponent()

    {

        

    }

    

    void timerCallback() override

    {

        c1.repaint();

        c2.repaint();

    };

    


    void paint (Graphics& g) override

    {

        g.fillAll(Colour(r.nextFloat(),1.f,0.5,1.f));

    }

    

    void resized() override

    {

        c1.setBounds(100,50,400,100);

        c2.setBounds(100,250,400,100);

    }

    

    class ChildComponent : public Component

    {

    public:

        ChildComponent() :

            r(334543)

        {

            setOpaque(true);

        };

        

        void paint (Graphics &g) override

        {

            g.fillAll(Colour(r.nextFloat(),1.f,0.5,1.f));

        };

        

        Random r;

    };



    ChildComponent c1;

    ChildComponent c2;

    Random r;


private:

    //==============================================================================

    JUCE_DECLARE_NON_COPYABLE_WITH_LEAK_DETECTOR (MainContentComponent)

};

http://i.imgur.com/9IhMB8V.png

2466×1160

 

1 Like
2

Oh my god, I'm so sick of this same question coming up again and again in different forms!

As I've explained countless times before:

CoreGraphics does not provide a way to ask whether a rectangle overlaps the dirty region that it's asking you to repaint. So the only way to implement CoreGraphicsContext::clipRegionIntersects() is to ask whether the rectangle overlaps any part of the entire clip bounds, and obviously that will provide some false positives. Areas of components that lie between dirty rectangles but which don't actually need drawing still need to have their paint methods called because we can't prove that they don't.

It's not a problem with other rendering engines. It's just a missing piece of functionality in CoreGraphics, and AFAIK there's no way to work around it.

3

Thanks for clarification!

Maybe you have inspected this, the documentation about is inconsistent.

The rectangle passed into your drawRect: routine is a union of the dirty rectangles, but applications running OS X version 10.3 and later can get a list of the individual rectangles, as described in Constraining Drawing to Improve Performance.

https://developer.apple.com/library/mac/documentation/Cocoa/Conceptual/CocoaViewsGuide/Optimizing/Optimizing.html

Furthermore:

In OS X version 10.3 and later, views can constrain their drawing even further by using the NSView methods getRectsBeingDrawn:count: and needsToDrawRect:. These methods provide direct and indirect access, respectively, to the detailed representation of a view’s invalid areas—that is, its list of non-overlapping rectangles—that the Application Kit maintains for each NSView instance. The Application Kit automatically enforces clipping to this list of rectangles, and you can further improve performance in views that do complex or expensive drawing by having them limit their drawing to objects that intersect any of the rectangles in this list.

 

 

While this is different form the new documentation page (which says needsToDrawRect checks only to the rect which is transmitted by drawRect)

needsToDrawRect

You typically send this message from within a drawRect: implementation. It gives you a convenient way to determine whether any part of a given graphical entity might need to be drawn. It is optimized to efficiently reject any rectangle that lies outside the bounding box of the area that the view is being asked to draw in drawRect:.

https://developer.apple.com/library/mac/documentation/Cocoa/Reference/ApplicationKit/Classes/NSView_Class/#//apple_ref/occ/instm/NSView/needsToDrawRect:

So maybe needsToDrawRect is the missing function?

 

 

 

 

4

Sure, it can get the original set of rectangles for the NSView, but the problem comes later on, as the CGContext that we're drawing into gets passed down the hierarchy of Components, and each one clips it in various ways, and saves/restores the state, applies affine transforms to it etc.

The only way this set of rectangles could be used to help would be if I implemented an entire system to keep track of the clip region in a way that mirrors the one that the CGContext already has internally. This'd be hugely complex and massively inefficient, as it'd add overhead to every graphics call.

5

Well, the way its done now,  it has massive overhead (just repaint to little controls, calls paint for the whole GUI)

Why not split up the drawRect calls directly in the JuceNSViewClass

  static void drawRect (id self, SEL, NSRect r)

    {

        if (NSViewComponentPeer* p = getOwner (self))

        {

            const NSRect* rects;

            NSInteger count;

            [self getRectsBeingDrawn:&rects count:&count];

            

            for (int i = 0; i < count; i++)

            {

                NSRect rc = rects[i];

                p->drawRect (rc);

            };

        };

    }

 

and then intersect the clip-context for every drawRect call in NSViewComponentPeer (inside save/restorrGraphicsState scope) 

its applied before the transforms, so i think it should be okay (?)

BTW: the clipping could be optional, if only one rect is provided

 

    void drawRect (NSRect r)

    {

        if (r.size.width < 1.0f || r.size.height < 1.0f)

            return;


        CGContextRef cg = (CGContextRef) [[NSGraphicsContext currentContext] graphicsPort];

        

        [NSGraphicsContext saveGraphicsState];  // New, don't forget to restore NSGraphicsContext down below

          CGContextClipToRect(cg, NSRectToCGRect(r));             // new


        if (! component.isOpaque())

            CGContextClearRect (cg, CGContextGetClipBoundingBox (cg));


        

        

        

        

        float displayScale = 1.0f;


       #if defined (MAC_OS_X_VERSION_10_7) && (MAC_OS_X_VERSION_MAX_ALLOWED >= MAC_OS_X_VERSION_10_7)

        NSScreen* screen = [[view window] screen];

        if ([screen respondsToSelector: @selector (backingScaleFactor)])

            displayScale = (float) screen.backingScaleFactor;

       #endif


       #if USE_COREGRAPHICS_RENDERING

        if (usingCoreGraphics)

        {

            CoreGraphicsContext context (cg, (float) [view frame].size.height, displayScale);


            insideDrawRect = true;

            handlePaint (context);

            insideDrawRect = false;

        }

        else

       #endif

        {

            const Point<int> offset (-roundToInt (r.origin.x),

                                     -roundToInt ([view frame].size.height - (r.origin.y + r.size.height)));

            const int clipW = (int) (r.size.width  + 0.5f);

            const int clipH = (int) (r.size.height + 0.5f);


            RectangleList<int> clip;

            getClipRects (clip, offset, clipW, clipH);


            if (! clip.isEmpty())

            {

                Image temp (component.isOpaque() ? Image::RGB : Image::ARGB,

                            roundToInt (clipW * displayScale),

                            roundToInt (clipH * displayScale),

                            ! component.isOpaque());


                {

                    const int intScale = roundToInt (displayScale);

                    if (intScale != 1)

                        clip.scaleAll (intScale);


                    ScopedPointer<LowLevelGraphicsContext> context (component.getLookAndFeel()

                                                                      .createGraphicsContext (temp, offset * intScale, clip));


                    if (intScale != 1)

                        context->addTransform (AffineTransform::scale (displayScale));


                    insideDrawRect = true;

                    handlePaint (*context);

                    insideDrawRect = false;

                }


                CGColorSpaceRef colourSpace = CGColorSpaceCreateDeviceRGB();

                CGImageRef image = juce_createCoreGraphicsImage (temp, colourSpace, false);

                CGColorSpaceRelease (colourSpace);

                CGContextDrawImage (cg, CGRectMake (r.origin.x, r.origin.y, clipW, clipH), image);

                CGImageRelease (image);

            }

        }

        

        [NSGraphicsContext restoreGraphicsState]; // new

    }


Edit: use of NSRectToCGRect

 

Edit: looks like it works here, very well (in a app wich also uses transformations)
 

1 Like
6

Jules - you probably don't have time to do this, but I'd be quite alright with reading technological brain-dump blogs of yours!

Such could be easily linked to posts like these. :)

7

<sigh>

I have thought about this, you know. For many years.

Yes, at the moment this "bug" causes some extra overhead some of the time, for some types of component layout.

With the rectangle-list trick I suggested above there'd be a big overhead all of the time, for every single graphics call, so that's a no-go.

Yes, per-rectangle rendering is an option, but doing that means that instead of there being one component graph traversal, there are N graph traversals, and N paint calls per component, and this is all very expensive to do. Years ago that's exactly how the original rendering code worked because back then, the juce graphics context class only had a rectangular clip region. When I rewrote it to allow complex clipping and made a single paint call, there was a huge speed improvement across apps in general, especially ones with complex component hierarchies or large numbers of components.

Saying "it works very well" is only looking at your own case, and your case is clearly unusual because of the fact that you've ended up here having a problem with it. Your suggestion above would ruin performance in many other people's existing apps, unfortunately.

1 Like
8

This is what we are doing since juce 1.53 (because we have a very heavy UI). Someone once, i can't find out anymore who, posted a solution where getRectsBeingDrawn is being used:

            CoreGraphicsContext context (cg, (float) [view frame].size.height, displayScale);


            #pragma mark BEGIN_MODIFIED_BY_US seems to speed up rendering

            bool onePaintForAll = false; //toggle this!

            if (onePaintForAll)
            {
                insideDrawRect = true;
                handlePaint (context);
                insideDrawRect = false; // original code

            } else {

                const NSRect* rects = 0;

                NSInteger numRects = 0;

                [view getRectsBeingDrawn: &rects count: &numRects];

                for (uint i = 0; i < numRects; i++)
                {

                    CGContextSaveGState(cg);
                    CGContextClipToRect(cg, CGRectMake(rects[i].origin.x, rects[i].origin.y, rects[i].size.width, rects[i].size.height));

                    insideDrawRect = true;
                    handlePaint (context);
                    insideDrawRect = false;
                    CGContextRestoreGState(cg);

                }

            }

            #pragma mark END_MODIFIED_BY_US


Set onePaintForAll to false to call paint for each rect. In the example chkn posted this will prevent the MainComponent paint to be called.
I modified the paint code to this to simulate a more heavy paint routine:

    g.fillAll(Colour(r.nextFloat(),1.f,0.5,1.f));
    g.setFont(getHeight()/2);

for (int i=0; i<100;++i)

        g.drawText("Test", i, i, getWidth()-i, getHeight()-i, Justification::centred);

I set the Timer to 1 ms and calculate fps. Original code runs at 45 fps, modified code runs at 95 fps. Of course this all depends whatever is painted, but it seems that our modification does speed things up quite a bit.
Note: with this change are UI is renderered quicker, but of course this might not be the case for every UI like Jules mentioned, i would say use it when it works for you. Add some methods to toggle it on/off to the NSViewComponentPeer and use it when applicable. 

 

9 
and your case is clearly unusual​​

This is the case when you have more than one opaque component (like VU-Meters, Spectral-meters), which are repainting very often. I would say this is very usual in a JUCE-plugin/application which has audio-functionality. So we can have 60fps instead of 24fps.

Why the text, between two VU-Meters, needs to be re-renderd every time, the VU-Meter is updated.

A dynamic flag is a good idea! What about a window-style flag, or something which is accessable from a plugin...

 

1 Like
10

deleted

11

Alternative 1:

If only opaque components are dirty, the rectangular list-method would be used, otherwise the single rectangular method is used.

Edit:

Whenever you call repaint() on a opqaue component, a counter will be incremented in the underlying NSView Component.

If the number equals with the number of rectangles which are reqeusted to redraw, the multi-rectangle method is used.

Alternative 2:

deleted

 

 

12

Here is Alternative 1 (proof of concept),  we can have both!

Quick opaque repainting, and single rectulanger behavior at once

 


diff --git a/modules/juce_gui_basics/components/juce_Component.cpp b/modules/juce_gui_basics/components/juce_Component.cpp
index 77c61c3..ef00385 100644
--- a/modules/juce_gui_basics/components/juce_Component.cpp
+++ b/modules/juce_gui_basics/components/juce_Component.cpp
@@ -1909,6 +1909,12 @@ void Component::internalRepaintUnchecked (Rectangle<int> area, const bool isEnti
 
             if (ComponentPeer* const peer = getPeer())
             {
+                
+                if (isOpaque())
+                {
+                    ++peer->numberOfOpaqueComponentsWhichNeedsToBeRepainted;
+                }
+                
                 // Tweak the scaling so that the component's integer size exactly aligns with the peer's scaled size
                 const Rectangle<int> peerBounds (peer->getBounds());
                 const Rectangle<int> scaled (area * Point<float> (peerBounds.getWidth()  / (float) getWidth(),
diff --git a/modules/juce_gui_basics/native/juce_mac_NSViewComponentPeer.mm b/modules/juce_gui_basics/native/juce_mac_NSViewComponentPeer.mm
index 69846cf..784944a 100644
--- a/modules/juce_gui_basics/native/juce_mac_NSViewComponentPeer.mm
+++ b/modules/juce_gui_basics/native/juce_mac_NSViewComponentPeer.mm
@@ -795,10 +795,33 @@ public:
         if (usingCoreGraphics)
         {
             CoreGraphicsContext context (cg, (float) [view frame].size.height, displayScale);
-
-            insideDrawRect = true;
-            handlePaint (context);
-            insideDrawRect = false;
+            
+            const NSRect* rects = 0;
+            NSInteger numRects = 0;
+            [view getRectsBeingDrawn: &rects count: &numRects];
+            
+            if (numberOfOpaqueComponentsWhichNeedsToBeRepainted.get()!=numRects)
+            {
+                insideDrawRect = true;
+                handlePaint (context);
+                insideDrawRect = false; // original code
+            } else
+            {
+                for (uint i = 0; i < numRects; i++)
+                {
+                    
+                    CGContextSaveGState(cg);
+                    CGContextClipToRect(cg, CGRectMake(rects[i].origin.x, rects[i].origin.y, rects[i].size.width, rects[i].size.height));
+                    
+                    insideDrawRect = true;
+                    handlePaint (context);
+                    insideDrawRect = false;
+                    CGContextRestoreGState(cg);
+                    
+                }
+            }
+            
+            numberOfOpaqueComponentsWhichNeedsToBeRepainted=0;
         }
         else
        #endif
diff --git a/modules/juce_gui_basics/windows/juce_ComponentPeer.h b/modules/juce_gui_basics/windows/juce_ComponentPeer.h
index 4277618..4eb0a56 100644
--- a/modules/juce_gui_basics/windows/juce_ComponentPeer.h
+++ b/modules/juce_gui_basics/windows/juce_ComponentPeer.h
@@ -355,6 +355,8 @@ public:
     virtual int getCurrentRenderingEngine() const;
     virtual void setCurrentRenderingEngine (int index);
 
+    Atomic<int> numberOfOpaqueComponentsWhichNeedsToBeRepainted;
+    
 protected:
     //==============================================================================
     Component& component;
@@ -369,6 +371,8 @@ private:
     const uint32 uniqueID;
     bool isWindowMinimised;
     Component* getTargetForKeyPress();
+    
+   
 
     JUCE_DECLARE_NON_COPYABLE_WITH_LEAK_DETECTOR (ComponentPeer)
 };
13

Just an update:

I spent a lot of time banging my head against this problem last week, but really can't see a good solution.

The ideas you've suggested seem ok, but every idea just fell apart when I looked at it more deeply. There are nasty edge-cases (the main one that I kept hitting is the case where you repaint an opaque component that is overlapped by a non-opaque sibling) that mean you can't just start the repaint from a child component, it always needs to walk the whole tree from the top-level component.

My gut feeling is that there just isn't a non-trivial hack that could be used here, because identifying when it's a trivial case is itself difficult.. But please keep the conversation going, any further suggestions are welcome!

1 Like
14

nasty edge-cases (the main one that I kept hitting is the case where you repaint an opaque component that is overlapped by a non-opaque sibling)

 

Thank for your efforts.

Both methods always guarantee proper repainting, right? 

The worst case for the one-clip rendering is that the paint()-method is called for a lot components which doesn't need to be repainted at all.

The worst case for multi-clip rendering is that a (whether or not) opaque component overlaps multiple redraw rects, and the same paint()-method is called multiple times.

I will think about it.

 

15

I would like to show you what we did to limit the stress caused by the UI repainting on OSX. What i'm about to show you is purely a trick we used to have more control over when and how often the UI repaints itself. It's certainly not a solution for everyone but i would like to share it anyway, who knows what it's good/

What we did is the following. We added a method to NSViewComponentPeer called setManualDisplay, this is what it does:

void NSViewComponentPeer::setManualDisplay(bool manual)
{
    [window setAutodisplay:!manual];
}

When you call the method above the Window's drawRect won't be called until you call the displayIfNeeded method on the View.
So in order to repaint your window you call performAnyPendingRepaintsNow on the peer.

void performAnyPendingRepaintsNow() override
{
     [view displayIfNeeded];
}

I have a timer in my MainWindow that call's display on the rate i choose. I call display 20 times a second but of course you can even throttle this based on your app's performance.

This trick is in our Application for a long time and it helped us tremendously in the past, i haven't touched it since so i'm not sure how much is needed still. But it's a simple and effective way to control the amount of paint calls.  
 

 

16

thanks, wouln't help here, this topic is more about unneccessay paint()-calls.

Maybe you have a rich-text layout between two VU-Meters, when both of them get repainted, the text-layout will be repainted too.

17

As long we have no better solution, a macro would be fine, to activate the mutiple rectangle redraw.

 

18

Any chance for a macro?

19

Added: JUCE_COREGRAPHICS_RENDER_WITH_MULTIPLE_PAINT_CALLS

3 Likes
20

THANKS! :-)