i am currently finetuning our OpenGL implementation and i have come accross a weird behaviour when resizing the plugin window which i cannot solve. It must be something deep in the juce code. When resizing the window all components seem to jump for a moment before getting back to their intended position. It seems to be a timing issue between some resizing operations and the update of the OpenGL content.
I have attached the simple plugin files where you can reproduce it easily, at least on Mac. The only edited files are those of the PluginEditor. Just create a new plugin in the projucer, add the OpenGL module and use those:
Basically this is what i changed from the original basic plugin created by the Projucer:
class NewProjectAudioProcessorEditor : public juce::AudioProcessorEditor, public juce::OpenGLRenderer
{
public:
NewProjectAudioProcessorEditor (NewProjectAudioProcessor&);
~NewProjectAudioProcessorEditor() override;
// OpenGLRenderer methods
void newOpenGLContextCreated() override {}
void openGLContextClosing() override {}
void renderOpenGL() override {}
private:
// This reference is provided as a quick way for your editor to
// access the processor object that created it.
NewProjectAudioProcessor& audioProcessor;
juce::OpenGLContext m_context;
juce::ImageComponent m_imc[100];
JUCE_DECLARE_NON_COPYABLE_WITH_LEAK_DETECTOR (NewProjectAudioProcessorEditor)
};
NewProjectAudioProcessorEditor::NewProjectAudioProcessorEditor (NewProjectAudioProcessor& p)
: AudioProcessorEditor (&p), audioProcessor (p)
{
// Make sure that before the constructor has finished, you've set the
// editor's size to whatever you need it to be.
setSize (600, 500);
setResizable(true, false);
m_context.setOpenGLVersionRequired(juce::OpenGLContext::OpenGLVersion::openGL3_2);
m_context.setRenderer (this);
m_context.setMultisamplingEnabled(true);
juce::OpenGLPixelFormat format;
format.multisamplingLevel = 4;
m_context.setPixelFormat(format);
m_context.setComponentPaintingEnabled ( true );
m_context.attachTo (*this);
m_context.setContinuousRepainting (true);
juce::Image im ( juce::Image::PixelFormat::ARGB, 400, 400, false );
juce::Graphics g ( im );
g.fillAll ( juce::Colours::red );
for ( int i = 0; i < 20; i++ )
{
m_imc[i].setImage ( im );
m_imc[i].setBounds ( 50, 50, 400, 400 );
addAndMakeVisible ( m_imc[i] );
}
}
NewProjectAudioProcessorEditor::~NewProjectAudioProcessorEditor()
{
m_context.detach();
}
I’ll try to reproduce at some point but it’s not top of my list right now. In the mean time if you have an easy to reproduce example it might be worth doing a bisect of JUCE versions to determine the commit that introduced the issue.
This thing is ages old. From before Juce 6 at least. I always struggled with this but now i took the time to boil away all my code to find that it is not caused by my code as you can see from the code example i posted.
My guess is that it was always like this. I dove into the juce code myself but i could not find the cause.
OpenGL rendering normally happens on a background thread, while window resizing happens on the main thread. However, on macOS, we attempt to paint in lockstep with the message thread during a live resize. That is, when the OpenGL component gets resized on the main thread, we update the viewport size and immediately render a new frame, rather than waiting to render the frame on the background render thread. I think this should mean that we always render at the correct size for the window, assuming the window always waits for the current frame to finish before getting displayed onscreen.
My theory is that painting with OpenGL occasionally misses the deadline for the current frame, and the OS decides to present whatever buffer it has ready, stretching or squashing the OpenGL view in order to keep the window resize feeling responsive. If this is indeed what’s happening, I doubt there’s anything we can do to change this behaviour.
As a quick test, I tried putting a std::this_thread::sleep_for() in OpenGLContext::CachedImage::handleResize() to simulate low framerates during resize. This makes the jumping/jittering much worse, despite the OpenGL rendering and flushBuffer calls being made synchronously.
Maybe there’s some OS API that I’ve missed to help improve this, or maybe I’m misusing the NSOpenGLContext API somehow - or maybe this is just how NSOpenGLView behaves during a resize. I remember investigating this issue in the past, and I’ve not had any insights looking at things with fresh eyes just now.
Unfortunately it could be very time consuming to even work out whether a fix is even viable, and I don’t see this happening any time soon, as we have quite a lot of other higher-priority work to get through. Sorry that this probably isn’t the answer you’re looking for.
Thanks for the feedback. I don’t think that it is justs the way OpenGL behaves. I am sure there is a way to properly synchronize things as other non-Juce software does this without any problem. Even with only one red square in my example code and built as release i can get it to jump. So i doubt that it is simply a performance issue.
Hi @reuk, in case it helps to add importance or give more context. I also just noticed the problem in windows, the openGl window context resizes much slower, even if it is a black window with no openGl elements.
I noticed doing my plugin and noticed these weird things that with juce 2d graphics doesn’t happen.
In the VST image, you can see lines in the background that should be covered by OpenGL, since they are made with JUCE graphics of the same size. However, the movement of the window prevents OpenGL from reacting properly, letting you see what should be hidden.
and in standalone it looks something like this, the effect expands more but I think it is enough to see the effect, where you can see a transparent separation without background:
although i understand that this is not an easy thing to fix or even investigate it still remains an annoying issue. On both OSes, Windows and Mac, resizing is far from smooth and other plugins which do not use juce but OpenGL or other means of rendering on the GPU resize smoothly. So it must be possible.
I had a look inside the rendering code and i saw that the OpenGL render call calls to a MessageManager lock. I might be wrong here but could this be the issue why there are artifacts even with very simple window content? It all seems to boil down to some kind of synch issue.
Nope, this is something only the juce team can solve. I know that there is a solution out there since there are plugins which use GPU accelerated gfx and window scaling with repositioning works smoothly, but digging deep in the OS specific code for synching OpenGL with the rest is currently beyond me.
Oh bummer! I find the glitches are very distracting.
They can be observed on the Demorunner, by the way. One just needs to select the OpenGLDemo2D, or OpenGL as renderer in the settings, and then resize the window. I’ve tested with earlier versions (5.1.0 and 6.1.0) with the same results.
It’s not something which happens only under heavy load and almost any setBounds() call will cause a visible “jump”. I’ve also found that the problem is even worse when setComponentPaintingEnabled is false.
It is basically a sync issue between the resizing and repositioning which happens on the main thread and the OpenGL rendering which happens on the OpenGL thread and the actual displaying of the OpenGL content on screen.
@reuk What would happen if one does not attempt to paint in lockstep during a resize? I guess it would lead to some other artifacts, but perhaps that’s preferable?
The lockstep does not solve the problem as there are still frames which escape. That causes the jumping. There are three threads which need to be synchronized:
Main thread: repositioning, repainting
OpenGL Render thread: rendering in OpenGL
Display thread: were everything gets displayed
When Main thread and OpenGL Render thread are locked, the display thread still runs free for some ms. That is where when resizing the OpenGL component the OpenGL rendered content gets distorted until a proper repositioning and repainting took place on the locked threads.
I see, thanks. I wonder how much these glitches vary from one system to the next. How is it on yours? On my ageing Intel-Mac with Sonoma it looks like this:
right now I just don’t render during resize which results in a black screen. Will probably update it to take a snapshot of the screen and then resize that. Maybe even turning OpenGL rendering on and off in that moment. Will update here if I have any success.
