Deadlock when using multiple OpenGLRenderer components as children of a non openGL component

opengl
#1

Hi,
I am trying to use two components that derive from OpenGLRenderer and each contain their own OpenGLContext. These components are each a child of a Component (that doesn’t derive from OpenGLRenderer of have an OpenGLContext).

This works in Windows and on 5 of 6 Mac OSX machines that I’ve tried. The one that has an issue doesn’t render any openGL and then hangs if I try to resize the window. It appears that the two openGL threads are both waiting for a lock while making a call to context.swapBuffers() in OpenGLContext::CachedImage::renderFrame().

I put together a simple GUI Application using the Projucer and can reproduce the problem on that same machine. I’ll include the relevant code I’m using below.

Does anyone know if this is possibly a known issue or if maybe I’m doing something wrong or missing something?

Thank you

class MyOpenGLComponent : public Component, public OpenGLRenderer
{
public:
    MyOpenGLComponent ()
    {
        openGLContext.setOpenGLVersionRequired (juce::OpenGLContext::OpenGLVersion::openGL3_2);
        openGLContext.setRenderer (this);
        openGLContext.setComponentPaintingEnabled (false);
        openGLContext.setMultisamplingEnabled (true);
        openGLContext.attachTo (*this);
    }

    ~MyOpenGLComponent ()
    {
        openGLContext.detach ();
    }

    //** Inherited from OpenGLRenderer **//
    virtual void newOpenGLContextCreated () override
    {
    }

    virtual void openGLContextClosing () override
    {
    }

    virtual void renderOpenGL () override
    {
        OpenGLHelpers::clear (juce::Colours::blanchedalmond);
    }

protected:
    OpenGLContext   openGLContext;
};

class MainComponent   : public Component
{
public:
    MainComponent()
    {
        ...
        addAndMakeVisible (myOpenGL1);
        addAndMakeVisible (myOpenGL2);
    }

    void resized()
    {
        Rectangle<int> bounds = getLocalBounds ();

        bounds.setHeight ((bounds.getHeight () / 2) - 20);
        myOpenGL1.setBounds (bounds);
        bounds.translate (0, bounds.getHeight () + 40);
        myOpenGL2.setBounds (bounds);
    }
    ...
private:
    MyOpenGLComponent myOpenGL1;
    MyOpenGLComponent myOpenGL2;
    ...
};
0 Likes

#2

In case its helpful here are the call stacks of the 2 openGL threads once they are in a deadlocked state:

2710 Thread_2712814: Pool
+ 2710 thread_start  (in libsystem_pthread.dylib) + 13  [0x1030aee61]
+   2710 _pthread_start  (in libsystem_pthread.dylib) + 70  [0x1030b2eff]
+     2710 _pthread_body  (in libsystem_pthread.dylib) + 126  [0x1030afe6d]
+       2710 juce::threadEntryProc(void*)  (in LAC) + 30  [0x100243d9e]  juce_posix_SharedCode.h:914
+         2710 juce::juce_threadEntryPoint(void*)  (in LAC) + 21  [0x10021e635]  juce_Thread.cpp:118
+           2710 juce::Thread::threadEntryPoint()  (in LAC) + 363  [0x10021e16b]  juce_Thread.cpp:96
+             2710 juce::ThreadPool::ThreadPoolThread::run()  (in LAC) + 66  [0x10028f3e2]  juce_ThreadPool.cpp:36
+               2710 juce::ThreadPool::runNextJob(juce::ThreadPool::ThreadPoolThread&)  (in LAC) + 337  [0x100221fc1]  juce_ThreadPool.cpp:383
+                 2710 juce::OpenGLContext::CachedImage::runJob()  (in LAC) + 299  [0x1006f9d0b]  juce_OpenGLContext.cpp:465
+                   2710 juce::OpenGLContext::CachedImage::renderFrame()  (in LAC) + 847  [0x1006fd01f]  juce_OpenGLContext.cpp:273
+                     2710 juce::OpenGLContext::swapBuffers()  (in LAC) + 44  [0x1006dce3c]  juce_OpenGLContext.cpp:971
+                       2710 juce::OpenGLContext::NativeContext::swapBuffers()  (in LAC) + 53  [0x1006dce85]  juce_OpenGL_osx.h:165
+                         2710 -[NSOpenGLContext flushBuffer]  (in AppKit) + 27  [0x7fff438f0ddc]
+                           2710 CGLFlushDrawable  (in OpenGL) + 59  [0x7fff500f6773]
+                             2710 glSwap_Exec  (in GLEngine) + 186  [0x7fff501149ea]
+                               2710 flush_notify  (in AppKit) + 110  [0x7fff43c586d1]
+                                 2710 _pthread_mutex_firstfit_lock_slow  (in libsystem_pthread.dylib) + 235  [0x1030ade86]
+                                   2710 _pthread_mutex_firstfit_lock_wait  (in libsystem_pthread.dylib) + 114  [0x1030b08e3]
+                                     2710 __psynch_mutexwait  (in libsystem_kernel.dylib) + 10  [0x7fff731aa872]
2710 Thread_2712816: Pool
+ 2710 thread_start  (in libsystem_pthread.dylib) + 13  [0x1030aee61]
+   2710 _pthread_start  (in libsystem_pthread.dylib) + 70  [0x1030b2eff]
+     2710 _pthread_body  (in libsystem_pthread.dylib) + 126  [0x1030afe6d]
+       2710 juce::threadEntryProc(void*)  (in LAC) + 30  [0x100243d9e]  juce_posix_SharedCode.h:914
+         2710 juce::juce_threadEntryPoint(void*)  (in LAC) + 21  [0x10021e635]  juce_Thread.cpp:118
+           2710 juce::Thread::threadEntryPoint()  (in LAC) + 363  [0x10021e16b]  juce_Thread.cpp:96
+             2710 juce::ThreadPool::ThreadPoolThread::run()  (in LAC) + 66  [0x10028f3e2]  juce_ThreadPool.cpp:36
+               2710 juce::ThreadPool::runNextJob(juce::ThreadPool::ThreadPoolThread&)  (in LAC) + 337  [0x100221fc1]  juce_ThreadPool.cpp:383
+                 2710 juce::OpenGLContext::CachedImage::runJob()  (in LAC) + 299  [0x1006f9d0b]  juce_OpenGLContext.cpp:465
+                   2710 juce::OpenGLContext::CachedImage::renderFrame()  (in LAC) + 847  [0x1006fd01f]  juce_OpenGLContext.cpp:273
+                     2710 juce::OpenGLContext::swapBuffers()  (in LAC) + 44  [0x1006dce3c]  juce_OpenGLContext.cpp:971
+                       2710 juce::OpenGLContext::NativeContext::swapBuffers()  (in LAC) + 53  [0x1006dce85]  juce_OpenGL_osx.h:165
+                         2710 -[NSOpenGLContext flushBuffer]  (in AppKit) + 27  [0x7fff438f0ddc]
+                           2710 CGLFlushDrawable  (in OpenGL) + 59  [0x7fff500f6773]
+                             2710 glSwap_Exec  (in GLEngine) + 186  [0x7fff501149ea]
+                               2710 flush_notify  (in AppKit) + 313  [0x7fff43c5879c]
+                                 2710 CGLGetSurface  (in OpenGL) + 55  [0x7fff500f11c0]
+                                   2710 _pthread_mutex_firstfit_lock_slow  (in libsystem_pthread.dylib) + 235  [0x1030ade86]
+                                     2710 _pthread_mutex_firstfit_lock_wait  (in libsystem_pthread.dylib) + 114  [0x1030b08e3]
+                                       2710 __psynch_mutexwait  (in libsystem_kernel.dylib) + 10  [0x7fff731aa872]
0 Likes

#3

My only advice would be to not use multiple juce::OpenGLContext objects, as they each spawn a thread and provide no synchronisation with each other.

They “usually” work out well on certain platforms like OSX, but I’ve found awful deadlocking issues (among other performance problems) on Linux and Windows platforms depending on your drivers.

The way we’re solving this is to always stick to only one instance of juce::OpenGLContext and achieve multiple juce::OpenGLRenderers by making a custom subclass of it (similar to you have) that is an opaque component. Setting the component to be opaque will allow the background OpenGL context to show through, since any juce::Graphics drawing will show above the juce::OpenGLRenderer graphics.

Note that you will still have issues if you’re making audio plugins, because many hosts don’t support running plugins as their own sub-process. When multiple plugins that use a juce::OpenGLContext run in a single host process, you will encounter the same issues as if you were using multiple contexts.

0 Likes

#4

Thank you for your advice.
It was convenient to have an OpenGLContext per area that I wanted to render OpenGL too, but if that results in synchronization issues between the OpenGL threads then I don’t think it’d be worth the possible issues that may randomly pop up (like this one I’m seeing now).

I think I understand what you’re suggesting. Only having one OpenGLContext means that I’ll have to have a component that at minimum is a union of all the areas I want to render OpenGL to. It will also have to be an OpenGLRenderer too. I have to make sure that I don’t call setComponentPaintingEnabled (false) for this component since it will need to be responsible for at least setting a background color in its paint call.
Then I’ll have to create child components that are OpenGLRenderers. These child components will need to call setOpaque(true) in order to prevent it’s parent’s paint function from drawing over it (this is needed because the OpenGL rendering is done first and then the paint call is made after).
The parent component will have to pass along the OpenGLRenderer calls to its children.
The last piece that I’ll need to consider is that the child component needs to know where it is located within the OpenGLContext. It will have to offset it’s OpenGL rendering to make sure it goes to the correct location within the OpenGLContext.

Does that sound about right? Am I missing or misunderstanding anything?

I’ll give this a try and see how it works.
Thanks again.

0 Likes

#5

It sounds about right, here’s the general steps for the approach I took:

  • We already had a “TopLevelWindow” class for our plugins that was used to handle certain settings like plugin size and such… this is the class I attached the OpenGLContext to

  • I updated our TopLevelWindow to add OpenGLRenderer to the classes it’s deriving from. I didn’t explicitly use setComponentPaintingEnabled(false) because we allow component painting still. HOWEVER, make sure that any components that “fill” the window (including this TopLevelWindow) use setOpaque(true). This is what allows the OpenGL rendering to show through when using the other components described below.

  • Made a RenderView class that inherits both OpenGLRenderer and Component. These are able to be positioned anywhere in the UI, essentially. This are also marked setOpaque(true). When these are opaque and the TopLevelWindow is opaque, you can skip drawing the RenderView background and the OpenGL rendering you’re doing will show through the component painting!

  • Added a Array<RenderView*> renderers member to the TopLevelWindow. When the OpenGLRenderer methods are called like newOpenGLContextCreated(), renderOpenGL(), openGLContextClosing() the TopLevelWindow really just iterates that array and calls those methods for each item

  • Before actually calling renderOpenGL(), we had to ensure that transparency was enabled (glEnable(GL_BLEND) and friends) and ensure that the viewport was set correctly. The viewport is based on supplying getLocalArea() with RenderView*, RenderView.getLocalBounds() and multiplying the result by OpenGLContext::getRenderingScale(). Our plugins allow for resizable windows, so we also had to take our own scale into account as well as that context scale.

  • OPTIONAL: Our plugins have features that require adding, removing, and re-ordering the renderers at run-time. Because these operations would come from the main thread, I had to essentially make two OpenGLRenderer* queues and add methods that atomically let you add/remove renderers. The methods use std::atomic_flag to ensure that only one thing is manipulating the queues at a time, letting us properly set up or tear down the target renderers. For example, if you were to add a renderer while the OpenGL loop is running, you couldn’t call its renderOpenGL() method until calling its newOpenGLContextCreated() method (assuming the renderer needs to allocate buffers etc., which is usually the case). Similarly, renderers in the removal queue have their renderOpenGL() callback called for the last time followed by openGLContextClosing() and are then subsequently removed from the renderers array.

  • OPTIONAL: Finally, RenderView used the same approach as above and held its own array/queues of OpenGLRenderer*. This is mostly because of the previous point: we’ll draw a grid, and RTA display, and an EQ curve all in the same view. If you don’t need that (i.e. each RenderView really just draws one thing) then you can just make a custom subclass of RenderView each time and write your OpenGL drawing code in the subclass

Sorry if that’s long winded :slight_smile: but let me know if you have any other questions and I’ll try to answer them as best as I can! The whole process was very annoying for me so I’d like to help people out in this regard.

1 Like

#6

I encountered some problems with multiple GL contexts too when I created some GL-Based realtime Plot Components. I added a class to share a GL context with multiple OpenGLRenderer instances. I have to admit that I don’t remember every detail of it, but it was essential to set the viewport for each component that implements some custom GL rendering correctly. Therefore my implementation is based on the assumption, that a Component that owns all Components that use GL rendering is defined as some top level component, the computation of the viewport coordinates relies on that.

You can find the shared GL context class here:


Here you see how I get the coordinates for the viewport

Here you see how I set the top level component needed for what I described above in an example project

Not sure if this really solves your problems but maybe it will help you getting started in some direction :wink:

1 Like

#7

@PluginPenguin very awesome stuff! Glad to see I’m not the only one who suffered through this issue :grin:

0 Likes

#8

Hello Tony,

this sounds bad. Does it mean, I cannot use an OpenGL context, in a plugin? Because my users will certainly use multiple instances of my plugin. Its an EQ. So it might very well get inserted into multiple tracks.
Whats the worst case scenario, users can encounter with this?

0 Likes

#9

Hey @Alatar,

Basically what we’ve found is this:

  • macOS is usually okay, although other users on the forum have reported performance issues regarding multiple contexts in the same process. We haven’t experienced the issues yet, nor have we ever experienced OpenGLContext related crashes on macOS

  • Windows is often okay, although we encountered lots of performance issues when running multiple plugins. We got around this by disabling vsync, but that hasn’t worked for other people apparently. I have encountered at least one sort of issue where the host froze up, but it was actually running our juce::OpenGLContext plugin at the same time with one of our older plugins which manages its own context using pugl (on the main thread)

  • Linux (which most people don’t support anyway it seems?) is basically the wild west. I’ve encountered lots of really awful issues on Linux with NVIDIA drivers when using multiple instances of juce::OpenGLContext. Sometimes even just trying to run a single instance wasn’t working on certain Linux setups. The nouveau drivers have seemed most reliable when running single instances of the context… but of course that means you’ll probably run into trouble as soon as you open a second copy of the plugin.

These issues do not appear on hosts that support running plugin instances in their own dedicated processes… but not all hosts support that. It also doesn’t mean the context will work anyway, as I’ve had quite a lot of trouble trying to get any of our plugins running on the Linux version of Bitwig (with dedicated plugin processes enabled).

This thread discusses the issues people have encountered in more detail:

Worst case scenario is the user’s host will completely freeze or crash, and if it doesn’t they may just run into performance issues and see some ugly FPS drops.

0 Likes

#10

Thanks Tony for your insights.

But you still use OpenGL, even though there are all those issues?

0 Likes

#11

We essentially have to, for our RTA displays.

What we haven’t done is taken the plunge into making a single-threaded (UI thread only) implementation of juce::OpenGLContext :slight_smile:

0 Likes