OpenGL: How do 3D with custom shaders and 2D with JUCE paint-methods work together?

1

I already asked a question regarding this topic this morning, that I decided to delete after having looked a bit deeper into the JUCE codebase.

I just made my first successful steps with OpenGL and the projucers OpenGL Application template. However as I plan to do something a bit more sophisticated I really would like to know how the rendering works under the hood.

The OpenGLAppComponent creates an OpenGLContext and then calls

openGLContext.setRenderer (this);
openGLContext.attachTo (*this);

So if I get it right, the first line tells the context to call OpenGLRenderer::renderOpenGL on a regular basis (which just straightly calls OpenGLAppComponent::render after incrementing the frame counter), while the second line tells it to perform any 2D painting calls via the OpenGL renderer instead of the CPU software-renderer, right?

Now what I don’t get is how the 2D painting actually works? If I’m creating an OpenGLShaderProgram to which I pass my shader, describing how to draw my 3D stuff in the render callback. I call OpenGLShaderProgram::use on my shader before performing any drawing. Is there a second shader program down in the JUCE codebase, that is invoked in a similar way before performing 2D painting? So, might even I provide multiple shader programs and use them sequentially during render to draw different stuff different?

Why am I interested in those details? Well, I’d like to create an OpenGL-driven 3D plot that should be a drop-in-replacement for a plot that’s currently done in 2D without using OpenGL. The rest of the 2D GUI should stay untouched. So I really want to understand how the combination of 2D and 3D elements work together.

3 Likes
2

No not exactly. You are right about setRenderer: this tells the context which OpenGLRenderer should be getting the opengl callbacks like the regular render callback that you mention. However, the attachTo method tells the OpenGLContext into which component to do it’s rendering, i.e. the gl commands that you execute in your render callback. Otherwise, how would JUCE know where to render those.

In addition, by default, as you mention, JUCE will also use OpenGL as a rendering engine for any JUCE graphics calls of the attached components and any of it’s children. However, you can also disable this via OpenGLContext::setComponentPaintingEnabled (false). Then JUCE will leave the rendering engine as it is. So primarily, the attachTo call is just a way to tell the gl context where it should display the output of any gl drawing commands.

Pretty much. If OpenGLContext::setComponentPaintingEnabled is enabled then JUCE will call the component’s paint callback (and all of it’s children) immediately after we call your OpenGLRenderer’s render callback. The Graphics object passed to the paint calls will use the OpenGLGraphicsContext instead of the default graphics engine and that context maps any JUCE drawing calls to OpenGL calls.

There are quite a few different shaders we use for this. You can find them here, here, here, here and many more in that file.

2 Likes
3

That cleared some things up, thank you!

Now let’s say that I want to create a UI with a Scope component and a Warerfall plot component, both OpenGL driven but rendered with very different shaders. Let’s also assume that both components have some 2D text and that the UI also has several other 2D components like sliders and buttons.

My idea would be to create two individual OpenGLContext instances for both the Scope and the Waterfall plot and then attach them to those. Then I’d create an OpenGLShaderProgram for each context - as said both quite different.

Is this approach with multiple OpenGLContext instances correct or is this the wrong way to go?

What I would expect would be:

  • The UI without my two special components but including all other stuff would be rendered by the software renderer
  • Each special component would be rendered by their own OpenGLContext using the ShaderProgram I created for each with their 2D painting being handled by the JUCE-supplied shaders in the same OpenGLContext (without me really noticing this as the API gently hides those details away from me)
  • Using the same font in the software-rendered UI parts and in the OpenGL-rendered components should result in the same look
4

In general the approach sounds good. However, OpenGLShaderProgram is quite limited. It’s probably better (and easier) to just load your own shader, attributes, vertex buffers, etc. in your OpenGLRenderer::newOpenGLContextCreated callback and then use them in your OpenGLRenderer::render callback - all just using standard gl commands (not JUCE).

Yes, if this is what you want. But why not just render everything in OpenGL? Then you can get away with just a single OpenGLContext on the top-level component. You then need to delegate the OpenGLRenderer callbacks to the two sub-components that want to do some gl rendering. If you go down this route then let me know. There are a few things you need to consider here and I’ll share some code to get you started.

Yup exactly.

Mmmmmhhh, probably not. There will be some differences in the way text is rendered when using different rendering engines.

Also note, that the JUCE 2d paint calls always draw on top of whatever you render in your OpenGLRenderer::render callback. But it seems that this is what you want anyways.

5

Nice to hear that the approach should basically work. While I’m still working my way through basic OpenGL topics this gives me a good idea on how everything works together with JUCE and what the end-product after having gotten super-familiar with OpenGL might look like. It’s always nice to have an idea of the whole thing when learning new things. In this context:

Even if I’m not that far at the moment, I’d really like to know how everything would work with a single OpenGLContext for everything - could be some useful information to me in a few weeks. So if you have time for sharing some code and knowledge this would be really nice!

6

please share it for the rest of us.

7

For the sake of argument, let’s say we have the following situation. A JUCE OpenGLContext is attached to the top-level component. Child components A and B want to do some low-level gl rendering (evaluate shaders, show spinning tea-pots, etc.) but want to use the top-level’s context for this.

56

So here are some of the things you need to worry about when trying to make this work:

1 . You can’t simply delegate the OpenGLRenderer callbacks of the top-level components to both A and B. This is because both A and B expect OpenGLRenderer::newOpenGLContextCreated/OpenGLRenderer::openGLContextClosing on initialisation/destruction. However, the top-level’s newOpenGLContextCreated will likely be called before A and B have even been constructed yet. There could also be more complicated setups where the construction of B happens much later: for example, if B only appears after some user interaction. Calling newOpenGLContextCreated from B’s constructor also doesn’t work as newOpenGLContextCreated must be called on the GL thread otherwise any gl command (like loading textures) will fail in newOpenGLContextCreated.
To make this work your top-level component needs a very simple work queue where it keeps track whose newOpenGLContextCreated/openGLContextClosing it needs to call. It checks this everytime it gets a render callback.

2 . You need to clip and transform the gl viewport when calling the render callback of A and B

3 . Dead-locking: if component A or B is deleted then it needs to free it’s opengl resources. However, A’s destructor is called on the message thread and opengl resources can only be released on the gl thread. The naïve approach would be to set a flag in the destructor and then wait in the destructor until the render callback is invoked again. B can then check the flag in the render callback and safely delete it’s opengl resources, setting another flag which tells the message thread that it can now unblock and continue destructing the component.
Unfortunately, this results in a dead-lock, as JUCE requires the message manager lock when rendering it’s own components via OpenGL. JUCE can’t get the lock though because you are blocking and waiting in B’s destructor.
To workaround this you can instruct JUCE to call a lambda on the gl thread without taking the message manager lock. You are not allowed to render anything in that lambda but you can safely delete/allocate resources.

Here is a class GLContextHolder I wrote a while back which will help you with the above. It’s currently used in ROLI internal code and I think it’s quite well tested.

You need to instantiate GLContextHolder as a member variable in your top-level component. It contains the OpenGLContext for you as a public member so the top-level component should not have another one. The top-level component must also explicitly call detach in it’s destructor before it deletes any children. A and B then register/deregister for OpenGLRenderer callbacks via registerOpnGlRenderer/unregisterOpenGlRenderer.

class GlContextHolder
		  : private juce::ComponentListener,
			private juce::OpenGLRenderer
{
public:
	static GlContextHolder* globalContextHolder;

	GlContextHolder (juce::Component& topLevelComponent)
		: parent (topLevelComponent)
	{
		jassert (globalContextHolder == nullptr);
		globalContextHolder = this;

		context.setRenderer (this);
		context.setContinuousRepainting (true);
		context.setComponentPaintingEnabled (true);
		context.attachTo (parent);
	}

	//==============================================================================
	// The context holder MUST explicitely call detach in their destructor
	void detach()
	{
		jassert (juce::MessageManager::getInstance()->isThisTheMessageThread());

		const int n = clients.size();
		for (int i = 0; i < n; ++i)
			if (juce::Component* comp = clients.getReference(i).c)
				comp->removeComponentListener (this);

		context.detach();
		context.setRenderer (nullptr);
	}

	//==============================================================================
	// Clients MUST call unregisterOpenGlRenderer manually in their destructors!!
	void registerOpenGlRenderer (juce::Component* child)
	{
		jassert (juce::MessageManager::getInstance()->isThisTheMessageThread());

		if (dynamic_cast<juce::OpenGLRenderer*> (child) != nullptr)
		{
			if (findClientIndexForComponent (child) < 0)
			{
				clients.add (Client (child, (parent.isParentOf (child) ? Client::State::running : Client::State::suspended)));
				child->addComponentListener (this);
			}
		}
		else
			jassertfalse;
	}

	void unregisterOpenGlRenderer (juce::Component* child)
	{
		jassert (juce::MessageManager::getInstance()->isThisTheMessageThread());

		const int index = findClientIndexForComponent (child);

		if (index >= 0)
		{
			Client& client = clients.getReference (index);
			{
				juce::ScopedLock stateChangeLock (stateChangeCriticalSection);
				client.nextState = Client::State::suspended;
			}

			child->removeComponentListener (this);
			context.executeOnGLThread ([this] (juce::OpenGLContext&)
									   {
										   checkComponents (false, false);
									   }, true);
			client.c = nullptr;

			clients.remove (index);
		}
	}

	void setBackgroundColour (const juce::Colour c)
	{
		backgroundColour = c;
	}

	juce::OpenGLContext context;

private:
	//==============================================================================
	void checkComponents (bool isClosing, bool isDrawing)
	{
		juce::Array<juce::Component*> initClients, runningClients;

		{
			juce::ScopedLock arrayLock (clients.getLock());
			juce::ScopedLock stateLock (stateChangeCriticalSection);

			const int n = clients.size();

			for (int i = 0; i < n; ++i)
			{
				Client& client = clients.getReference (i);
				if (client.c != nullptr)
				{
					Client::State nextState = (isClosing ? Client::State::suspended : client.nextState);

					if      (client.currentState == Client::State::running   && nextState == Client::State::running)   runningClients.add (client.c);
					else if (client.currentState == Client::State::suspended && nextState == Client::State::running)   initClients   .add (client.c);
					else if (client.currentState == Client::State::running   && nextState == Client::State::suspended)
					{
						dynamic_cast<juce::OpenGLRenderer*> (client.c)->openGLContextClosing();
					}

					client.currentState = nextState;
				}
			}
		}

		for (int i = 0; i < initClients.size(); ++i)
			dynamic_cast<juce::OpenGLRenderer*> (initClients.getReference (i))->newOpenGLContextCreated();

		if (runningClients.size() > 0 && isDrawing)
		{
			const float displayScale = static_cast<float> (context.getRenderingScale());
			const juce::Rectangle<int> parentBounds = (parent.getLocalBounds().toFloat() * displayScale).getSmallestIntegerContainer();

			for (int i = 0; i < runningClients.size(); ++i)
			{
				juce::Component* comp = runningClients.getReference (i);

				juce::Rectangle<int> r = (parent.getLocalArea (comp, comp->getLocalBounds()).toFloat() * displayScale).getSmallestIntegerContainer();
				glViewport ((GLint) r.getX(),
							(GLint) parentBounds.getHeight() - (GLint) r.getBottom(),
							(GLsizei) r.getWidth(), (GLsizei) r.getHeight());
				juce::OpenGLHelpers::clear (backgroundColour);

				dynamic_cast<juce::OpenGLRenderer*> (comp)->renderOpenGL();
			}
		}
	}

	//==============================================================================
	void componentParentHierarchyChanged (juce::Component& component) override
	{
		if (Client* client = findClientForComponent (&component))
		{
			juce::ScopedLock stateChangeLock (stateChangeCriticalSection);

			client->nextState = (parent.isParentOf (&component) && component.isVisible() ? Client::State::running : Client::State::suspended);
		}
	}

	void componentVisibilityChanged (juce::Component& component) override
	{
		if (Client* client = findClientForComponent (&component))
		{
			juce::ScopedLock stateChangeLock (stateChangeCriticalSection);

			client->nextState = (parent.isParentOf (&component) && component.isVisible() ? Client::State::running : Client::State::suspended);
		}
	}

	void componentBeingDeleted (juce::Component& component) override
	{
		const int index = findClientIndexForComponent (&component);

		if (index >= 0)
		{
			Client& client = clients.getReference (index);

			// You didn't call unregister before deleting this component
			jassert (client.nextState == Client::State::suspended);
			client.nextState = Client::State::suspended;

			component.removeComponentListener (this);
			context.executeOnGLThread ([this] (juce::OpenGLContext&)
									   {
										   checkComponents (false, false);
									   }, true);

			client.c = nullptr;

			clients.remove (index);
		}
	}

	//==============================================================================
	void newOpenGLContextCreated() override
	{
		checkComponents (false, false);
	}

	void renderOpenGL() override
	{
		juce::OpenGLHelpers::clear (backgroundColour);
		checkComponents (false, true);
	}

	void openGLContextClosing() override
	{
		checkComponents (true, false);
	}

	//==============================================================================
	juce::Component& parent;

	struct Client
	{
		enum class State
		{
			suspended,
			running
		};

		Client (juce::Component* comp, State nextStateToUse = State::suspended)
			 : c (comp), currentState (State::suspended), nextState (nextStateToUse) {}


		juce::Component* c = nullptr;
		State currentState = State::suspended, nextState = State::suspended;
	};

	juce::CriticalSection stateChangeCriticalSection;
	juce::Array<Client, juce::CriticalSection> clients;

	//==============================================================================
	int findClientIndexForComponent (juce::Component* comp) const
	{
		const int n = clients.size();
		for (int i = 0; i < n; ++i)
			if (comp == clients.getReference (i).c)
				return i;

		return -1;
	}

	Client* findClientForComponent (juce::Component* comp) const
	{
		const int index = findClientIndexForComponent (comp);
		if (index >= 0)
			return &clients.getReference (index);

		return nullptr;
	}

	//==============================================================================
	juce::Colour backgroundColour { juce::Colours::black };
};
11 Likes
8

Yeeesh can’t click that heart icon enough times for that code!!!

9

Hi @fabian, is there an updated version of the OpenGLContextHolder class? I just wanted to make sure before diving into troubleshooting. Many thanks!!