Inconsistencies in OpenGLGraphicsContextCustomShader


#1

Hi,

 

Here's the Juce's OpenGL 2D demo modified to attach a texture uniform to the shader.

/*

  ==============================================================================


   This file is part of the JUCE library.

   Copyright (c) 2015 - ROLI Ltd.


   Permission is granted to use this software under the terms of either:

   a) the GPL v2 (or any later version)

   b) the Affero GPL v3


   Details of these licenses can be found at: www.gnu.org/licenses


   JUCE is distributed in the hope that it will be useful, but WITHOUT ANY

   WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR

   A PARTICULAR PURPOSE.  See the GNU General Public License for more details.


   ------------------------------------------------------------------------------


   To release a closed-source product which uses JUCE, commercial licenses are

   available: visit www.juce.com for more information.


  ==============================================================================

*/


#include "../JuceDemoHeader.h"


#if JUCE_OPENGL



//==============================================================================

class OpenGL2DShaderDemo  : public Component,

                            private CodeDocument::Listener,

                            private ComboBox::Listener,

                            private Timer

{

public:

    OpenGL2DShaderDemo()

        : fragmentEditorComp (fragmentDocument, nullptr)

    {

        setOpaque (true);

        MainAppWindow::getMainAppWindow()->setOpenGLRenderingEngine();


        addAndMakeVisible (statusLabel);

        statusLabel.setJustificationType (Justification::topLeft);

        statusLabel.setColour (Label::textColourId, Colours::black);

        statusLabel.setFont (Font (14.0f));


        Array<ShaderPreset> presets (getPresets());

        StringArray presetNames;


        for (int i = 0; i < presets.size(); ++i)

            presetBox.addItem (presets[i].name, i + 1);


        addAndMakeVisible (presetLabel);

        presetLabel.setText ("Shader Preset:", dontSendNotification);

        presetLabel.attachToComponent (&presetBox, true);


        addAndMakeVisible (presetBox);

        presetBox.addListener (this);


        Colour editorBackground (Colours::white.withAlpha (0.6f));

        fragmentEditorComp.setColour (CodeEditorComponent::backgroundColourId, editorBackground);

        fragmentEditorComp.setOpaque (false);

        fragmentDocument.addListener (this);

        addAndMakeVisible (fragmentEditorComp);


        presetBox.setSelectedItemIndex (0);

    }


    ~OpenGL2DShaderDemo()

    {

        shader = nullptr;

    }


    void paint (Graphics& g)

    {

        g.fillCheckerBoard (getLocalBounds(), 48, 48, Colours::lightgrey, Colours::white);


        OpenGLContext * context = OpenGLContext::getCurrentContext();

        if (shader == nullptr || shader->getFragmentShaderCode() != fragmentCode)

        {

            shader = nullptr;


            if (fragmentCode.isNotEmpty())

            {

                shader = new OpenGLGraphicsContextCustomShader (fragmentCode);


                Result result (shader->checkCompilation (g.getInternalContext()));


                if (result.failed())

                {

                    statusLabel.setText (result.getErrorMessage(), dontSendNotification);

                    shader = nullptr;

                }

                else

                {   // Worked, let's bind an uniform for the texture

                    if (!textureUniform)

                    {

                        textureUniform = new OpenGLShaderProgram::Uniform (*shader->getProgram(g.getInternalContext()), "demoTexture");

                        texture.release();

                        texture.loadImage(resizeImageToPowerOfTwo(ImageFileFormat::loadFrom (BinaryData::portmeirion_jpg, BinaryData::portmeirion_jpgSize)));

                    }

                }

            }

        }


        if (shader != nullptr)

        {

            statusLabel.setText (String::empty, dontSendNotification);


                        context->extensions.glActiveTexture (GL_TEXTURE0);

                        glEnable (GL_TEXTURE_2D);

                        texture.bind();

                        glTexParameteri (GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_REPEAT);

                        glTexParameteri (GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_REPEAT);

                        shader->getProgram(g.getInternalContext())->use();

                        textureUniform->set((GLint)0);


            shader->fillRect (g.getInternalContext(), getLocalBounds());

            

              texture.unbind();

              glDisable(GL_TEXTURE_2D);

        }

    }


    void resized() override

    {

        Rectangle<int> area (getLocalBounds().reduced (4));


        statusLabel.setBounds (area.removeFromTop (75));


        area.removeFromTop (area.getHeight() / 2);


        Rectangle<int> presets (area.removeFromTop (25));

        presets.removeFromLeft (100);

        presetBox.setBounds (presets.removeFromLeft (150));


        area.removeFromTop (4);

        fragmentEditorComp.setBounds (area);

    }


    void selectPreset (int preset)

    {

        fragmentDocument.replaceAllContent (getPresets()[preset].fragmentShader);

        startTimer (1);

    }


    ScopedPointer<OpenGLGraphicsContextCustomShader> shader;


    Label statusLabel, presetLabel;

    ComboBox presetBox;

    CodeDocument fragmentDocument;

    CodeEditorComponent fragmentEditorComp;

    String fragmentCode;

    OpenGLTexture texture;

    ScopedPointer<OpenGLShaderProgram::Uniform> textureUniform;


private:

    enum { shaderLinkDelay = 500 };


    void codeDocumentTextInserted (const String& /*newText*/, int /*insertIndex*/) override

    {

        startTimer (shaderLinkDelay);

    }


    void codeDocumentTextDeleted (int /*startIndex*/, int /*endIndex*/) override

    {

        startTimer (shaderLinkDelay);

    }


    void timerCallback() override

    {

        stopTimer();

        fragmentCode = fragmentDocument.getAllContent();

        repaint();

    }


    void comboBoxChanged (ComboBox*) override

    {

        selectPreset (presetBox.getSelectedItemIndex());

    }


    struct ShaderPreset

    {

        const char* name;

        const char* fragmentShader;

    };


    static Image resizeImageToPowerOfTwo (Image image)

    {

        if (! (isPowerOfTwo (image.getWidth()) && isPowerOfTwo (image.getHeight())))

            return image.rescaled (jmin (1024, nextPowerOfTwo (image.getWidth())),

                                   jmin (1024, nextPowerOfTwo (image.getHeight())));


        return image;

    }


    static Array<ShaderPreset> getPresets()

    {

        #define SHADER_DEMO_HEADER \

            "/*  This demo shows the use of the OpenGLGraphicsContextCustomShader,\n" \

            "    which allows a 2D area to be filled using a GL shader program.\n" \

            "\n" \

            "    Edit the shader program below and it will be \n" \

            "    recompiled in real-time!\n" \

            "*/\n\n"


        ShaderPreset presets[] =

        {

            {

                "Texture based",


                SHADER_DEMO_HEADER

                "uniform sampler2D demoTexture;\n"

                "void main()\n"

                "{\n"

                "    gl_FragColor = texture2D (demoTexture, gl_FragCoord.xy / 1000.0);\n"

                "}\n"

            },


            {

                "Simple Gradient",


                SHADER_DEMO_HEADER

                "void main()\n"

                "{\n"

                "    " JUCE_MEDIUMP " vec4 colour1 = vec4 (1.0, 0.4, 0.6, 1.0);\n"

                "    " JUCE_MEDIUMP " vec4 colour2 = vec4 (0.0, 0.8, 0.6, 1.0);\n"

                "    " JUCE_MEDIUMP " float alpha = pixelPos.x / 1000.0;\n"

                "    gl_FragColor = pixelAlpha * mix (colour1, colour2, alpha);\n"

                "}\n"

            },


            {

                "Circular Gradient",


                SHADER_DEMO_HEADER

                "void main()\n"

                "{\n"

                "    " JUCE_MEDIUMP " vec4 colour1 = vec4 (1.0, 0.4, 0.6, 1.0);\n"

                "    " JUCE_MEDIUMP " vec4 colour2 = vec4 (0.3, 0.4, 0.4, 1.0);\n"

                "    " JUCE_MEDIUMP " float alpha = distance (pixelPos, vec2 (600.0, 500.0)) / 400.0;\n"

                "    gl_FragColor = pixelAlpha * mix (colour1, colour2, alpha);\n"

                "}\n"

            },


            {

                "Circle",


                SHADER_DEMO_HEADER

                "void main()\n"

                "{\n"

                "    " JUCE_MEDIUMP " vec4 colour1 = vec4 (0.1, 0.1, 0.9, 1.0);\n"

                "    " JUCE_MEDIUMP " vec4 colour2 = vec4 (0.0, 0.8, 0.6, 1.0);\n"

                "    " JUCE_MEDIUMP " float distance = distance (pixelPos, vec2 (600.0, 500.0));\n"

                "\n"

                "    " JUCE_MEDIUMP " float innerRadius = 200.0;\n"

                "    " JUCE_MEDIUMP " float outerRadius = 210.0;\n"

                "\n"

                "    if (distance < innerRadius)\n"

                "        gl_FragColor = colour1;\n"

                "    else if (distance > outerRadius)\n"

                "        gl_FragColor = colour2;\n"

                "    else\n"

                "        gl_FragColor = mix (colour1, colour2, (distance - innerRadius) / (outerRadius - innerRadius));\n"

                "\n"

                "    gl_FragColor *= pixelAlpha;\n"

                "}\n"

            },


            {

                "Solid Colour",


                SHADER_DEMO_HEADER

                "void main()\n"

                "{\n"

                "    gl_FragColor = vec4 (1.0, 0.6, 0.1, pixelAlpha);\n"

                "}\n"

            }

        };


        return Array<ShaderPreset> (presets, numElementsInArray (presets));

    }


    JUCE_DECLARE_NON_COPYABLE_WITH_LEAK_DETECTOR (OpenGL2DShaderDemo)

};


//==============================================================================

// This static object will register this demo type in a global list of demos..

static JuceDemoType<OpenGL2DShaderDemo> demo ("20 Graphics: OpenGL 2D");


#endif

It might be useful for other (as I've seen requests on the forum).

However, there are numerous issues with this code. I get a lot of assertion failed in checkGLError code, internal to Juce's graphic context (unrelated to fillRect). I've added a check of GL error code right after any of my change but could not get anything.

I'm not sure why, but you must "use()" the program before setting the uniform. You also need to enable a lot of GL stuff before calling fillRect, and if you disable them afterward, you get asserts in Juce's code.

More importantly, you can not release the texture, since you've no callback when/before the OpenGL context is destructed (it's 2D code I'm not subclassing the OpenGLRenderer like in the official 3D OpenGL demo).  

How to do that ?

As I understand this, a change to the OpenGLGraphicsContextCustomShader is required. You might add an operation queue interface that would be in charge of calling the binding code for your own stuff, the unbinding code, and also the context destructing code.

Ideally, you should not leak any GL state between as it leads to errors later on in the GL state machine.

Something like this:

struct Operation 
{
    /** This is called before drawing the fragment */
    virtual bind(OpenGLContext & context, OpenGLProgramShader & shader) = 0;
    /** This is called after drawing the fragment */
    virtual unbind(OpenGLContext & context, OpenGLProgramShader & shader) = 0;
    /** This is called just before the OpenGLContext is destructed (you might free your resources here) */
    virtual contextDestructing(OpenGLContext & context) = 0;
}; 
// In your OpenGLGraphicsContextCustomShader, you'll need an array/queue/whatever of those, and call the methods at the right time

What do you think ?


#2

*Bump*

Right now, OpenGLGraphicsContextCustomShader is a toy for a demo but nothing else (you can not do anything serious with it as it is).

We can not add any attribute to the shader program (because it interacts with the "hidden" pixelAlpha and useless pixelPos attribute), and we can not release the texture if we add a uniform to the program because we have no way to release it.

 

 

 


#3

*up*


#4

Yes, saw this, will get onto it asap!


#5

Cool - would be great.  Be good for my temporary gradient dithering fill routine too ;-)


#6

Re: releasing resources when the context is destroyed, that's what the OpenGLContext::get/setAssociatedObject methods are for. They're already used for things like CPU-cached texture versions of images, etc and seem to work pretty well. I wouldn't want to create a different class that does the same job.


#7

Ok, that would solve the texture not-destroyed error, but how about the other issues ?

Typically, Juce's code is bind/unbinding many things, and if I do my homework on my side (bind, use, unbind), I would expect that this does not break later Juce's code, but it does. At some point in time, we'll bind/unbind the same thing many time (or in a incompatible way), it's going to be non-optimal, and full of errors...

For example, if I want to add a uniform for the texture size, currently I can't do that properly (since I don't know when to unbind, and what'll be the ID of the uniform since Juce's using some uniform internally). I need to give Juce's my "OpenGL" objects, and let it bind/unbind at its own will.

 

 


#8

To add my two cents: I think your code is still missing the binding of the texture and setting the uniform to the right value (0):

// in paint()
            texture.bind();
            textureUniform->set (0);
            shader->fillRect (g.getInternalContext(), getLocalBounds());
            texture.unbind();

But, yes, ultimately your point remains valid: there is no way to bind things before the fillRect and no way to unbind after the fillRect call. However, at this point that's a limitation of the OpenGLGraphicsContextCustomShader: you can only use shaders which do not depend on the current state of the context. This means that there is no way to currently use textures in the custom shader. It's worth thinking about how we could fix this in the future.


Requested improvements to OpenGL shader support in JUCE
Requested improvements to OpenGL shader support in JUCE
#9

@fabian @jules has OpenGLGraphicsContextCustomShader received any attention since it was written? Particularly the limitations expressed in this thread?