I noticed some really sluggish scrolling on the MacBook pro Retina in my smooth-scrolling Viewport. I tracked down the problem to Graphics::drawImageAt(). Drawing a fullscreen image seems to make everything drop to 5 fps or so. Not drawing the image, only drawing text, filled rectangles and rectangles, doesn’t hurt the performance.
I haven’t looked at JUCE’s sources a lot, but it seems by default, when instantiating the Image class, a NativeImage is created, so it should actually reside on the GPU RAM and therefore be quickly displayed? Any ideas?
Hmm. The NativeImage class is really at the mercy of the OS as to where the image data is stored - it does everything in a way that allows it to go on the GPU, but whether it does so or not is up to CoreGraphics…
The problem also arises when using the Software Renderer with big components being repainted - since the whole area of the component will be blitted at some point, you get stalls there. Isn’t there maybe another way to transfer the image to the GPU, maybe another format, or what do I know … ?
All I can tell you is that in Windows 7, with the same machine, the same problem does not arise. It’s as if Windows didn’t suffer from this slow blitting problem (I’m not saying it’s super-fast either, since we’re talking about a 2880x1800 resolution, but it’s still very acceptable).
Maybe, but it’s not my code that does the transfer, it’s the OS!
Surprising that Windows would be faster, I’d have expected that it’d all use the same mechanism.
If you could put together a simple non-JUCE application that demonstrates the problem then you could float it around #macdev on Freenode.
#macdev folks report that NSImage is not going to be particularly fast.
Did you profile it to make sure the problem is with drawing the image?
The consensus is that for scrolling images, a “layer backed view” via CoreAnimation is the right approach:
There’s a CoreImage bug that Juce hits. I think Jules had a workaround at one point. It occurs at certain zoom ratios - you can duplicate it in JuceDemo. Maybe it’s that?
JuceDemo, graphics page, CoreGraphics renderer:
Turn on all animations,
go to tiled ARGB imges (swear I’ve hit it on other image types too)
Bruce
I’ve done some tests. Basically I have a fullscreen app that has a single Viewport in which a big component with a big image (bigger then screen) is in. When I scroll the Viewport’s ScrollBars, I get these results (note that CoreGraphics rendering has no “blitting” therefore it’s just one number for the CoreGraphics renderer).
software rendering (full screen), NSHighResolutionCapable off
- drawing image +/- 5ms
- blitting time: 1ms-20ms
remark: still acceptable
software rendering (full screen), NSHighResolutionCapable on
- drawing image +/- 5ms
- blitting time: 1ms-80ms
CoreGraphics rendering (full screen), NSHighResolutionCapable off
- total drawing time in NSViewComponentPeer::drawRect() : +/-15ms
remark: funnily enough, scrolling seems already very slow
CoreGraphics rendering (full screen), NSHighResolutionCapable on
- total drawing time in NSViewComponentPeer::drawRect() : +/-100ms
remark: scrolling is completely unusable, seems like 5fps or so
extra test: CoreGraphics rendering (full screen), NSHighResolutionCapable on, drawing a big ellipse instead of image: total time about 30ms
I should note that I’ve found out using my own framework (which uses software rendering) that it is possible to blit a fullscreen image of 2880x1800 onto the screen in about 30ms (and rendering is possible in much less than that via software rendering, for instance it takes 2ms to fill a 2880x1800 image with black colour).
I seems again that CoreGraphics is very slow
As a second note: I’ve modified JUCE so the software renderer also works on Retina. For anybody, who’s interested, I’ve attached the JUCE demo with it. You’ll see that the OpenGL renderer is best, but it doesn’t support Retina (is this hard to add?). The CoreGraphics renderer is sometimes slower then the software renderer, but mostly not.
https://www.dropbox.com/s/je3zo69uznu6i8u/JuceDemo.zip <- modified JUCE with Retina software renderer
https://www.dropbox.com/s/sel712ulx9hyi72/framework_retina.zip <- test with my own framework to check blitting speed
Have you tried disabling antialiasing (if active)?
Yes, disabled interpolation and antialiasing. Just all the same. On the same machine, in Windows, the blitting is lightning fast.
If this was in a stand-alone repo I could try to coax the #macdev folks into diagnosing it.
FYI: I’m on the case with the GL stuff, will check in later…
If this was in a stand-alone repo I could try to coax the #macdev folks into diagnosing it.[/quote]
There’s not much to say, the code that blits looks like follows (also tried NSDrawBitmap, just as slow).
[code]void blitBackBuffer(const unsigned char *buffer, int x, int y, int w, int h, int linestride)
{
CGContextRef cg = (CGContextRef) [[NSGraphicsContext currentContext] graphicsPort];
CGColorSpaceRef colourSpace = CGColorSpaceCreateDeviceRGB();
CGDataProviderRef provider = CGDataProviderCreateWithData (0, buffer, linestride * h, 0);
const bool interpolate = false;
CGImageRef imageRef = CGImageCreate (w, h,
8, 32, linestride,
colourSpace, kCGBitmapByteOrderDefault, provider,
0, interpolate, kCGRenderingIntentDefault);
CGDataProviderRelease (provider);
CGColorSpaceRelease (colourSpace);
const int retinaMultiplier = 2; // 2 for retina, 1 for non-retina
CGContextDrawImage (cg, CGRectMake (x/retinaMultiplier, y/retinaMultiplier, w/retinaMultiplier, h/retinaMultiplier), imageRef);
CGImageRelease (imageRef);
}[/code]
The last possible thing I can think of is to switch to OpenGL for uploading texture & blitting it. Might be faster, but I’d be surprised.
Good news! It seems that OpenGL is also much faster when it comes to blitting images on the MacBook Pro Retina!
Following code just creates a 2880x1800 Image, then fills it with some data and finally draws it…
[code]class MyComponent:public Component, public Timer
{
public:
MyComponent()
{
averageTime=0.0;
startTimer(20);
}
void paint(Graphics &g)
{
double t1 = Time::getMillisecondCounterHiRes();
g.fillAll(Colours::black);
int w=2880;
int h=1800;
Image image(Image::RGB, w, h, false);
Image::BitmapData data(image, 0, 0, w, h, Image::BitmapData::readWrite);
uint8* p=data.getPixelPointer(0,0);
int k=w*h*4;
for (int i=0; i<k; i++)
p[i]=(uint8)i;
int retinaMultiplier=2;
g.drawImage (image, 0, 0, w/retinaMultiplier, h/retinaMultiplier, 0, 0, w, h, false);
double t2 = Time::getMillisecondCounterHiRes();
averageTime=averageTime*0.9+(t2-t1)*0.1;
g.drawText(String(averageTime), 0, 300, 100, 20, Justification::centred, false);
}
void timerCallback()
{
repaint();
}
private:
double averageTime;
};[/code]
Here the results:
MacBook Pro Retina (from 2012) (2880x1800 image):
CoreGraphics: 120ms
OpenGLRenderer: 38ms
iMac 7,1 (from 2006) (1920x1200 image):
CoreGraphics: 23ms
OpenGLRenderer: 95ms
MacBook Pro 7.1 (from 2010) (1280x800 image):
CoreGraphics: 6ms
OpenGLRenderer: 47ms
which means that OpenGL is 3x (!!) faster in blitting then CoreGraphics on the MacBook Pro Retina, but CoreGraphics is better on the iMac 7,1 and MacBook Pro 7.1.
If JUCE’s software renderer will also be adapted to Retina-capabilities (I can help Jules with that as I’ve already done it), then it would be wise to offer the option to not blit via CoreGraphics, but with OpenGL instead. JUCE could automatically find out which of them is faster by doing some test before startup.
1 Like
Find attached a screenshot of the Time Profiler results. Maybe anybody can point out why the highlighted function is taking so much time…
I found one article about this function: http://robert.ocallahan.org/2010/05/cglayer-performance-trap-with-isflipped_03.html but in my case, wether NSView returns isFlipped YES or NO doesn’t change anything.
Here’s another profiling screenshot (CoreGraphics renderer with Graphics::drawImage() ).
I do not understand what is going on here: It clearly reports that from 79%-11,3% = 67,7% of the whole CPU time is eaten by MyComponent::paint(), but the few % attributed to the functions inside do not sum up to such a big number! It’s the first time I see this with the profiler, can anybody tell me what this is supposed to mean?
I have similar issues, but on iOS (iPad 3).
Drawing big images is incredibly slow and if I want to scroll a fullscreen image, the app drops down to 3 fps… :?
Using OpenGL doesn’t help, in fact I get even worse results.
Any ideas?
Has there been any progress regarding this matter? Either with CG or as Jules said above, with OpenGL. Anything that speeds that up would be great.
It is a really bad problem if retina images are used and renders the whole framework unusable.
Here is a post that explains what could be going on, and a possible solution if still desired to use CoreGraphics:
http://stackoverflow.com/questions/10410106/drawing-image-with-coregraph...
Thanks.
Seems like CoreGraphics allocates memory in the draw operation, and memory allocation is blocking
This is a really old thread, guys, and much of it is now defunct - am going to close it, please start a new topic elsewhere if there’s some more to discuss!