Deadlock when using multiple OpenGLRenderer components as children of a non openGL component

Hi,
I am trying to use two components that derive from OpenGLRenderer and each contain their own OpenGLContext. These components are each a child of a Component (that doesn’t derive from OpenGLRenderer of have an OpenGLContext).

This works in Windows and on 5 of 6 Mac OSX machines that I’ve tried. The one that has an issue doesn’t render any openGL and then hangs if I try to resize the window. It appears that the two openGL threads are both waiting for a lock while making a call to context.swapBuffers() in OpenGLContext::CachedImage::renderFrame().

I put together a simple GUI Application using the Projucer and can reproduce the problem on that same machine. I’ll include the relevant code I’m using below.

Does anyone know if this is possibly a known issue or if maybe I’m doing something wrong or missing something?

Thank you

class MyOpenGLComponent : public Component, public OpenGLRenderer
{
public:
    MyOpenGLComponent ()
    {
        openGLContext.setOpenGLVersionRequired (juce::OpenGLContext::OpenGLVersion::openGL3_2);
        openGLContext.setRenderer (this);
        openGLContext.setComponentPaintingEnabled (false);
        openGLContext.setMultisamplingEnabled (true);
        openGLContext.attachTo (*this);
    }

    ~MyOpenGLComponent ()
    {
        openGLContext.detach ();
    }

    //** Inherited from OpenGLRenderer **//
    virtual void newOpenGLContextCreated () override
    {
    }

    virtual void openGLContextClosing () override
    {
    }

    virtual void renderOpenGL () override
    {
        OpenGLHelpers::clear (juce::Colours::blanchedalmond);
    }

protected:
    OpenGLContext   openGLContext;
};

class MainComponent   : public Component
{
public:
    MainComponent()
    {
        ...
        addAndMakeVisible (myOpenGL1);
        addAndMakeVisible (myOpenGL2);
    }

    void resized()
    {
        Rectangle<int> bounds = getLocalBounds ();

        bounds.setHeight ((bounds.getHeight () / 2) - 20);
        myOpenGL1.setBounds (bounds);
        bounds.translate (0, bounds.getHeight () + 40);
        myOpenGL2.setBounds (bounds);
    }
    ...
private:
    MyOpenGLComponent myOpenGL1;
    MyOpenGLComponent myOpenGL2;
    ...
};
1 Like

In case its helpful here are the call stacks of the 2 openGL threads once they are in a deadlocked state:

2710 Thread_2712814: Pool
+ 2710 thread_start  (in libsystem_pthread.dylib) + 13  [0x1030aee61]
+   2710 _pthread_start  (in libsystem_pthread.dylib) + 70  [0x1030b2eff]
+     2710 _pthread_body  (in libsystem_pthread.dylib) + 126  [0x1030afe6d]
+       2710 juce::threadEntryProc(void*)  (in LAC) + 30  [0x100243d9e]  juce_posix_SharedCode.h:914
+         2710 juce::juce_threadEntryPoint(void*)  (in LAC) + 21  [0x10021e635]  juce_Thread.cpp:118
+           2710 juce::Thread::threadEntryPoint()  (in LAC) + 363  [0x10021e16b]  juce_Thread.cpp:96
+             2710 juce::ThreadPool::ThreadPoolThread::run()  (in LAC) + 66  [0x10028f3e2]  juce_ThreadPool.cpp:36
+               2710 juce::ThreadPool::runNextJob(juce::ThreadPool::ThreadPoolThread&)  (in LAC) + 337  [0x100221fc1]  juce_ThreadPool.cpp:383
+                 2710 juce::OpenGLContext::CachedImage::runJob()  (in LAC) + 299  [0x1006f9d0b]  juce_OpenGLContext.cpp:465
+                   2710 juce::OpenGLContext::CachedImage::renderFrame()  (in LAC) + 847  [0x1006fd01f]  juce_OpenGLContext.cpp:273
+                     2710 juce::OpenGLContext::swapBuffers()  (in LAC) + 44  [0x1006dce3c]  juce_OpenGLContext.cpp:971
+                       2710 juce::OpenGLContext::NativeContext::swapBuffers()  (in LAC) + 53  [0x1006dce85]  juce_OpenGL_osx.h:165
+                         2710 -[NSOpenGLContext flushBuffer]  (in AppKit) + 27  [0x7fff438f0ddc]
+                           2710 CGLFlushDrawable  (in OpenGL) + 59  [0x7fff500f6773]
+                             2710 glSwap_Exec  (in GLEngine) + 186  [0x7fff501149ea]
+                               2710 flush_notify  (in AppKit) + 110  [0x7fff43c586d1]
+                                 2710 _pthread_mutex_firstfit_lock_slow  (in libsystem_pthread.dylib) + 235  [0x1030ade86]
+                                   2710 _pthread_mutex_firstfit_lock_wait  (in libsystem_pthread.dylib) + 114  [0x1030b08e3]
+                                     2710 __psynch_mutexwait  (in libsystem_kernel.dylib) + 10  [0x7fff731aa872]
2710 Thread_2712816: Pool
+ 2710 thread_start  (in libsystem_pthread.dylib) + 13  [0x1030aee61]
+   2710 _pthread_start  (in libsystem_pthread.dylib) + 70  [0x1030b2eff]
+     2710 _pthread_body  (in libsystem_pthread.dylib) + 126  [0x1030afe6d]
+       2710 juce::threadEntryProc(void*)  (in LAC) + 30  [0x100243d9e]  juce_posix_SharedCode.h:914
+         2710 juce::juce_threadEntryPoint(void*)  (in LAC) + 21  [0x10021e635]  juce_Thread.cpp:118
+           2710 juce::Thread::threadEntryPoint()  (in LAC) + 363  [0x10021e16b]  juce_Thread.cpp:96
+             2710 juce::ThreadPool::ThreadPoolThread::run()  (in LAC) + 66  [0x10028f3e2]  juce_ThreadPool.cpp:36
+               2710 juce::ThreadPool::runNextJob(juce::ThreadPool::ThreadPoolThread&)  (in LAC) + 337  [0x100221fc1]  juce_ThreadPool.cpp:383
+                 2710 juce::OpenGLContext::CachedImage::runJob()  (in LAC) + 299  [0x1006f9d0b]  juce_OpenGLContext.cpp:465
+                   2710 juce::OpenGLContext::CachedImage::renderFrame()  (in LAC) + 847  [0x1006fd01f]  juce_OpenGLContext.cpp:273
+                     2710 juce::OpenGLContext::swapBuffers()  (in LAC) + 44  [0x1006dce3c]  juce_OpenGLContext.cpp:971
+                       2710 juce::OpenGLContext::NativeContext::swapBuffers()  (in LAC) + 53  [0x1006dce85]  juce_OpenGL_osx.h:165
+                         2710 -[NSOpenGLContext flushBuffer]  (in AppKit) + 27  [0x7fff438f0ddc]
+                           2710 CGLFlushDrawable  (in OpenGL) + 59  [0x7fff500f6773]
+                             2710 glSwap_Exec  (in GLEngine) + 186  [0x7fff501149ea]
+                               2710 flush_notify  (in AppKit) + 313  [0x7fff43c5879c]
+                                 2710 CGLGetSurface  (in OpenGL) + 55  [0x7fff500f11c0]
+                                   2710 _pthread_mutex_firstfit_lock_slow  (in libsystem_pthread.dylib) + 235  [0x1030ade86]
+                                     2710 _pthread_mutex_firstfit_lock_wait  (in libsystem_pthread.dylib) + 114  [0x1030b08e3]
+                                       2710 __psynch_mutexwait  (in libsystem_kernel.dylib) + 10  [0x7fff731aa872]

My only advice would be to not use multiple juce::OpenGLContext objects, as they each spawn a thread and provide no synchronisation with each other.

They “usually” work out well on certain platforms like OSX, but I’ve found awful deadlocking issues (among other performance problems) on Linux and Windows platforms depending on your drivers.

The way we’re solving this is to always stick to only one instance of juce::OpenGLContext and achieve multiple juce::OpenGLRenderers by making a custom subclass of it (similar to you have) that is an opaque component. Setting the component to be opaque will allow the background OpenGL context to show through, since any juce::Graphics drawing will show above the juce::OpenGLRenderer graphics.

Note that you will still have issues if you’re making audio plugins, because many hosts don’t support running plugins as their own sub-process. When multiple plugins that use a juce::OpenGLContext run in a single host process, you will encounter the same issues as if you were using multiple contexts.

2 Likes

Thank you for your advice.
It was convenient to have an OpenGLContext per area that I wanted to render OpenGL too, but if that results in synchronization issues between the OpenGL threads then I don’t think it’d be worth the possible issues that may randomly pop up (like this one I’m seeing now).

I think I understand what you’re suggesting. Only having one OpenGLContext means that I’ll have to have a component that at minimum is a union of all the areas I want to render OpenGL to. It will also have to be an OpenGLRenderer too. I have to make sure that I don’t call setComponentPaintingEnabled (false) for this component since it will need to be responsible for at least setting a background color in its paint call.
Then I’ll have to create child components that are OpenGLRenderers. These child components will need to call setOpaque(true) in order to prevent it’s parent’s paint function from drawing over it (this is needed because the OpenGL rendering is done first and then the paint call is made after).
The parent component will have to pass along the OpenGLRenderer calls to its children.
The last piece that I’ll need to consider is that the child component needs to know where it is located within the OpenGLContext. It will have to offset it’s OpenGL rendering to make sure it goes to the correct location within the OpenGLContext.

Does that sound about right? Am I missing or misunderstanding anything?

I’ll give this a try and see how it works.
Thanks again.

1 Like

It sounds about right, here’s the general steps for the approach I took:

  • We already had a “TopLevelWindow” class for our plugins that was used to handle certain settings like plugin size and such… this is the class I attached the OpenGLContext to

  • I updated our TopLevelWindow to add OpenGLRenderer to the classes it’s deriving from. I didn’t explicitly use setComponentPaintingEnabled(false) because we allow component painting still. HOWEVER, make sure that any components that “fill” the window (including this TopLevelWindow) use setOpaque(true). This is what allows the OpenGL rendering to show through when using the other components described below.

  • Made a RenderView class that inherits both OpenGLRenderer and Component. These are able to be positioned anywhere in the UI, essentially. This are also marked setOpaque(true). When these are opaque and the TopLevelWindow is opaque, you can skip drawing the RenderView background and the OpenGL rendering you’re doing will show through the component painting!

  • Added a Array<RenderView*> renderers member to the TopLevelWindow. When the OpenGLRenderer methods are called like newOpenGLContextCreated(), renderOpenGL(), openGLContextClosing() the TopLevelWindow really just iterates that array and calls those methods for each item

  • Before actually calling renderOpenGL(), we had to ensure that transparency was enabled (glEnable(GL_BLEND) and friends) and ensure that the viewport was set correctly. The viewport is based on supplying getLocalArea() with RenderView*, RenderView.getLocalBounds() and multiplying the result by OpenGLContext::getRenderingScale(). Our plugins allow for resizable windows, so we also had to take our own scale into account as well as that context scale.

  • OPTIONAL: Our plugins have features that require adding, removing, and re-ordering the renderers at run-time. Because these operations would come from the main thread, I had to essentially make two OpenGLRenderer* queues and add methods that atomically let you add/remove renderers. The methods use std::atomic_flag to ensure that only one thing is manipulating the queues at a time, letting us properly set up or tear down the target renderers. For example, if you were to add a renderer while the OpenGL loop is running, you couldn’t call its renderOpenGL() method until calling its newOpenGLContextCreated() method (assuming the renderer needs to allocate buffers etc., which is usually the case). Similarly, renderers in the removal queue have their renderOpenGL() callback called for the last time followed by openGLContextClosing() and are then subsequently removed from the renderers array.

  • OPTIONAL: Finally, RenderView used the same approach as above and held its own array/queues of OpenGLRenderer*. This is mostly because of the previous point: we’ll draw a grid, and RTA display, and an EQ curve all in the same view. If you don’t need that (i.e. each RenderView really just draws one thing) then you can just make a custom subclass of RenderView each time and write your OpenGL drawing code in the subclass

Sorry if that’s long winded :slight_smile: but let me know if you have any other questions and I’ll try to answer them as best as I can! The whole process was very annoying for me so I’d like to help people out in this regard.

6 Likes

I encountered some problems with multiple GL contexts too when I created some GL-Based realtime Plot Components. I added a class to share a GL context with multiple OpenGLRenderer instances. I have to admit that I don’t remember every detail of it, but it was essential to set the viewport for each component that implements some custom GL rendering correctly. Therefore my implementation is based on the assumption, that a Component that owns all Components that use GL rendering is defined as some top level component, the computation of the viewport coordinates relies on that.

You can find the shared GL context class here:


Here you see how I get the coordinates for the viewport

Here you see how I set the top level component needed for what I described above in an example project

Not sure if this really solves your problems but maybe it will help you getting started in some direction :wink:

3 Likes

@PluginPenguin very awesome stuff! Glad to see I’m not the only one who suffered through this issue :grin:

Hello Tony,

this sounds bad. Does it mean, I cannot use an OpenGL context, in a plugin? Because my users will certainly use multiple instances of my plugin. Its an EQ. So it might very well get inserted into multiple tracks.
Whats the worst case scenario, users can encounter with this?

Hey @Alatar,

Basically what we’ve found is this:

  • macOS is usually okay, although other users on the forum have reported performance issues regarding multiple contexts in the same process. We haven’t experienced the issues yet, nor have we ever experienced OpenGLContext related crashes on macOS

  • Windows is often okay, although we encountered lots of performance issues when running multiple plugins. We got around this by disabling vsync, but that hasn’t worked for other people apparently. I have encountered at least one sort of issue where the host froze up, but it was actually running our juce::OpenGLContext plugin at the same time with one of our older plugins which manages its own context using pugl (on the main thread)

  • Linux (which most people don’t support anyway it seems?) is basically the wild west. I’ve encountered lots of really awful issues on Linux with NVIDIA drivers when using multiple instances of juce::OpenGLContext. Sometimes even just trying to run a single instance wasn’t working on certain Linux setups. The nouveau drivers have seemed most reliable when running single instances of the context… but of course that means you’ll probably run into trouble as soon as you open a second copy of the plugin.

These issues do not appear on hosts that support running plugin instances in their own dedicated processes… but not all hosts support that. It also doesn’t mean the context will work anyway, as I’ve had quite a lot of trouble trying to get any of our plugins running on the Linux version of Bitwig (with dedicated plugin processes enabled).

This thread discusses the issues people have encountered in more detail:

Worst case scenario is the user’s host will completely freeze or crash, and if it doesn’t they may just run into performance issues and see some ugly FPS drops.

2 Likes

Thanks Tony for your insights.

But you still use OpenGL, even though there are all those issues?

We essentially have to, for our RTA displays.

What we haven’t done is taken the plunge into making a single-threaded (UI thread only) implementation of juce::OpenGLContext :slight_smile:

Thank you @TonyAtHarrison and @PluginPenguin. Your posts were both very helpful. I went with the approach of creating a few base objects to try and encapsulate everything so the rest of the code would just derive from these base objects and work the same whether they were running within their own context or not (very similar it seems to what PluginPenguin did).

I was struggling with getting the coordinates correct before realizing that glViewport was the key to getting everything scaled and translated to the correct location. After correctly using glViewport then everything else (coordinates related) just worked as if it was using its own context!

Thank you again, I was really struggling with this one and getting some validation and direction was just what I needed!

3 Likes

I am in the process of implementing a single OpenGLContext based rendering system like the one described by @TonyAtHarrison. Tony, your description above is extremely helpful, but I have a question. You stated that to add/remove renderers at run-time, you had to create some kind of synchronization system since these requests would be run on the Message thread while the GL thread could also be accessing the renderer queues.

Is this synchronization unnecessary since the OpenGL thread holds a MessageManager::Lock while calling renderOpenGL?

There was a previous post I made asking if accessing Message thread data (Component width/height) in renderOpenGL() was a race condition, but I was told it was not because the GL thread holds a Message thread lock whenever it calls renderOpenGL():

From what I understand, if you were adding, removing, or re-ordering renderers at run-time from the message thread in a system as you describe, @TonyAtHarrison:

  • When adding a new renderer at runtime, you could call the renderer’s newOpenGLContextCreated() method and then immediately add it to the array of renderers.
  • When removing a renderer, call its openGLContextClosing() method and immediately remove it from the array of renderers
  • When re-ordering, you would simply just swap various positions in the array of renderers.
  • And all of this functionality could happen without needing any synchronization such as locks or atomic variables.

I may be completely misunderstanding this, but would love to know if my thinking here is wrong so I can correct my assumptions. Thanks for any help here!

@adamski Is there any feedback you could provide regarding your WindowGLContext and my last question in this thread? (`WindowGLContext code here on GitHub)

Since the OpenGL thread always holds a MessageManager::Lock when calling renderOpenGL(), wouldn’t there be no need to lock when adding or removing new render targets?

For example, say I have an OpenGLRenderer oscilloscope that I want to add to a single instance of your WindowGLContext windowContext. Somewhere, from the Message thread, I might call:

windowContext->addRenderingTarget (oscilloscope);

When addRenderingTarget is called on the message thread, regardless of whether or not the OpenGL thread is running or not, wouldn’t there be no need for addRenderingTarget to lock it’s array of rendering targets (juce::Array<juce::OpenGLRenderer*>) since the GL thread holds a Message Manager lock?

Thanks for any help here! Just trying to clarify my understanding. And thank you again for this code, it has been extremely helpful in implementing a solution for using a single OpenGLContext with multiple OpenGLRenderers!

Sorry for bumping an old thread, but I’m having trouble getting this working.

I think I understand everything in @alassandro 's reply, but I feel like I’m missing something when it comes to having to set the renderer for the context. If I try to change the renderer while still attached to TopLevelWindow I keep hitting a jassert. And constantly attaching and reattching the context to TopLevelWindow is super slow.

Are the children ‘Renderers’ not actually acting as OpenGLRenderer but more so just instructions to provide to the context?