Unit testing basics

[Aueh]

Im trying to unit test a basic function in the prepareToPlay function of a basic audioplugin project. That is here.

void TestPluginAudioProcessor::prepareToPlay (double sampleRate, int samplesPerBlock)
{
    // Use this method as the place to do any pre-playback
    // initialisation that you need..
    delayBuffer.setSize(getTotalNumOutputChannels(), (int)(sampleRate * 2));
    readPosition = negativeModulo(writePosition - delayTime * 44100, delayBuffer.getNumSamples());
    
    
    UnitTestRunner* runner;
    Test test;
    test.performTest(runner);
}

Theres probably alot of things wrong with this, but just started with Juce and im just trying to get stuff on the screen. When i run the debug this pops up, from the juce_threads_windows.cpp file.

I have two files regarding tests.
Test.cpp:

#include "Test.h"


void Test::runTest()
{
    beginTest("begin");

    expect(TestPluginAudioProcessor::negativeModulo(-5, 10) == 5);
}

and Test.h:

#pragma once
#include <JuceHeader.h>
#include "PluginProcessor.h"

class  Test : public UnitTest
{
public:
    Test() : UnitTest("negativemodulo testing") {}
    void runTest() override;
};

They are in project as the plugin project and ive done #include "Test.h"
I cant find any tutorials on using the unit test features of Juce. So im not really sure what to do.

[anthony-nicholls]

It’s a little odd to run tests in something like prepareToPlay(), normally there would be a whole separate application (normally a command line application) that you would build and run your tests in (see the UnitTestRunner in extras as an example).

That being said the code probably doesn’t work because you haven’t actually created a UnitTestRunner, you’ve declared a pointer to one, but it doesn’t actually point to an instance.

In your example it would make more sense to write (warning see below, don’t actually do this, and I haven’t tested it either)

UnitTestRunner runner;
Test test;
test.performTest (&runner);

However, the documentation for performTest() says…

You shouldn’t need to call this method directly - use UnitTestRunner::runTests() instead.

So I think the way you would actually write this is something like (again untested)…

UnitTestRunner runner;
Test test;
runner.runTests ({ &test });

However it’s worth noting that when any UnitTest instance is created it’s automatically registered and so you should be able to run the test like so…

UnitTestRunner runner;
Test test;
runner.runAllTests();

JUCE takes advantage of this throughout the whole framework, every unit test class that is declared is followed by a static instance, something like…

class SomeUnitTest final : public UnitTest
{
public:
    SomeUnitTest() : UnitTest ("SomeUnitTest", UnitTestCategories::someCategory) {}

    void runTest() override
    {
        beginTest ("some test");
        {
            // create some variable or state
            // test some preconditions
            // do something
            // test the post conditions
        }
    }
};

// This is the static instance that is automatically registered so that a call 
// anywhere in the code base to `UnitTest::getAllTests()` will include this test

static SomeUnitTest someUnitTest;

If you search around the code base for JUCE_UNIT_TESTS you should be able to find all the tests, then look for the UnitTestRunner as an example of how they can be run.

Hope that helps.

[Aueh]

Thanks, so much useful information! I cant look at it now, because school, but i have a good feeling im gonna make progress.

[Aueh]

Ive tried to understand the https://github.com/juce-framework/JUCE/tree/master/extras/UnitTestRunner as an example and ive looked at the unit tests in the code base. What ive understood is that the unit tests are defined in the plugin project and the unittestrunner is used in a console project. So thats what ive tried.
Ive made a tests.cpp file in the plugin project.

#include "PluginProcessor.h"

class  Test final : public UnitTest
{
public:
    Test() : UnitTest("negativemodulo testing") {}

    void runTest() override;
};

static const Test test;

void Test::runTest()
{
    beginTest("begin");

    expect(TestPluginAudioProcessor::negativeModulo(-5, 10) == 5);
}

And i made a console project from projucer of which the Main.cpp file looks like this.

#include <JuceHeader.h>

//this is the file in the plugin project where my unit tests are defined
#include "../../TestPlugin/Source/tests.cpp" 

//==============================================================================
int main (int argc, char* argv[])
{

    // ..your code goes here!

    UnitTestRunner runner;
    Test test;
    runner.runTests({ &test });

    return 0;
}

Now note if i run this code in the prepareToPlay function, the tests are actually being run so thats good atleast. But if i run the console application i run in to the problem that my console application does not have the dependencies my plugin has. So i get all the corresponding errors when i run the console app. So does that mean that i need to add all the dependencies of the plugin to my test project? Either way thats what ive tried, but i ran into a new problem. This meant i also had to add the module juce_audio_plugin_client because thats what the plugin project relies on and so does my console app need to. But then i get some error saying that i need to pick atleast one plugin format. But this is a console app, it doesnt have a plugin format. I mean how do i pick console project and at the same time pick a plugin format.
I dont understand because the unittestrunner example does not rely on any of those dependencies. Maybe its because it uses runAllTests() instead of runTests(). For runAllTests() you dont need to instantiate unittest objects. But i dont really want to use it, because i dont want to run all registered unit tests when i just want to run a small test.

[Aueh]

Ive been trying all morning to just get a console app to run my unit test. This is my process:
I found the UnitTestsDemo in the repo, this is an app which can run unittests for you. It looks like this.

I hoped somehow that atleast this app can pick up my test. So following the direction in the window, i used the JUCE_UNIT_TESTS macro. And i also set JUCE_UNIT_TESTS=0 in the preprocessor definitions of project containing the unit test. It should be 0 afaik, i read it somewhere but i forgot where. Anyway setting it to 1 doesnt change anything.

#pragma once
#include "PluginProcessor.h"

#if JUCE_UNIT_TESTS
class  Test : public UnitTest
{
public:
    Test() : UnitTest("myTest") {}

    void runTest() override;
};

static Test test;

void Test::runTest()
{
    beginTest("begin");

    expect(TestPluginAudioProcessor::negativeModulo(-5, 10) == 5);
}
static Test test;
#endif

As you can see i also declared a static instance, so that UnitTest::getAllTests() should find my test.
I also added a bit of code to the UnitTestsDemo.h file:

UnitTestsDemo()
{
    setOpaque (true);

    addAndMakeVisible (startTestButton);
    startTestButton.onClick = [this] { start(); };

    addAndMakeVisible (testResultsBox);
    testResultsBox.setMultiLine (true);
    testResultsBox.setFont (Font (Font::getDefaultMonospacedFontName(), 12.0f, Font::plain));

    addAndMakeVisible (categoriesBox);
    categoriesBox.addItem ("All Tests", 1);

    auto categories = UnitTest::getAllCategories();
    categories.sort (true);
    
    /////////////////// MY ADDITION//////////////////////////
    for (auto* test : UnitTest::getAllTests())
        if (test->getName() == "myTest")
            logMessage(test->getName());
    //////////////////////////////////////////////////////////////////////
    categoriesBox.addItemList (categories, 2);
    categoriesBox.setSelectedId (1);

    logMessage ("This panel runs the built-in JUCE unit-tests from the selected category.\n");
    logMessage ("To add your own unit-tests, see the JUCE_UNIT_TESTS macro.");

    setSize (500, 500);
}

But unfortunately is it not found. All the examples run the framework unit tests, but adding your own seems really hard. And then even if i do get the system to pick up my unit test i probably will run into some dependency conflicts.

[anthony-nicholls]

If JUCE_UNIT_TESTS is 0 then any thing inside an #if JUCE_UNIT_TESTS will NOT compile.

So for example…

#if JUCE_UNIT_TESTS
 #error "JUCE_UNIT_TESTS is enabled"
#else
 #error "JUCE_UNIT_TESTS is NOT enabled"
#endif

In the above code if JUCE_UNIT_TESTS is 0 (or not defined) a compiler error will be thrown saying “JUCE_UNIT_TESTS is NOT enabled” if it’s anything else it should print “JUCE_UNIT_TESTS is enabled”.

The UnitTestsDemo app can only run tests that are compiled into that application, so if you want to include your own tests you’ll have to include those source files in the UnitTestsDemo project.

To make your life easier here’s what I would do…

• Open the Projucer
• Select File >> New Project
• Select Console (under Application) on the left hand side panel
• Enter a project name on the right
• Leave everything else in the default state for now
• Select Create Project... in the bottom right
• Select File >> Save Project and Open in IDE... (or click the IDE icon at the top of the application)
• Check that the project compiles and runs
• Replace the contents of main.cpp with the file below

#include <JuceHeader.h>

class MyFirstUnitTest final : public juce::UnitTest
{
public:
    MyFirstUnitTest() : juce::UnitTest ("MyFirstUnitTest") {}

    void runTest()
    {
        beginTest ("My first test");
        {
            expect (false);
        }
    }
};

static MyFirstUnitTest myUnitTest;

//==============================================================================
int main()
{
    juce::UnitTestRunner runner;
    runner.runAllTests();
    return 0;
}

I’ve tried to keep the above as simple as possible there are a number of improvements that could be made but it should demonstrate the basic point.

• Compile and run the application
• You should hit a failing test! this is good!
• Lets fix the test, change expect (false); to expect (true);
• Compile and run the application

OK at this point you should have a working console application for running your own unit tests, but the next step is to get your own tests into this application.

To do that you’ll need to

• Add all the source and header files containing both your test code and the code those tests rely on to the Projucer project
• Add all the JUCE modules that your source code depends on

These last two points should be well covered in the Projucer tutorials on the website

• JUCE: Tutorial: Projucer Part 1: Getting started with the Projucer
• JUCE: Tutorial: Projucer Part 2: Manage your Projucer projects

I hope that helps.

[Aueh]

Thank you for the reply. I did everything you suggested, but i got an error i was afraid of.

And i assume this is because the console app now uses the audio_plugin_client module. And since i picked console project there are some variables from the audio_plugin_client module that remain uninitialised, i guess. And i need this dependency because the file with the code i want to test also contains code that depends on the audio_plugin_client. But ok i understand that i can separate the code i want to test from all the code which is incompatible with a console app, and put that code in a separate file. But what if at some point i want to test more complicated objects, then its gonna be hard to not have any dependency with plugin components. It seems i will have to do a lot of patchwork just to get some tests running.

[anthony-nicholls]

Writing tests does tend to force the code structure to be a little different. IME this is often to the benefit of the code as it tends to encourage you to break your code up into more modular testable chunks which in turn makes your code more flexible to change in the future. It’s also important to note that you don’t need to test everything!

It’s important to note that a “UnitTest” should test a “unit” which ideally should be well isolated. I would suggest that any dependencies it has should be injected or the object itself should be used to compose other objects which have little to no logic and therefore probably won’t benefit from testing in the same way.

In your case I can see you have an AudioProcessor with what appears to be a static function called negativeModulo, given this is a static function this can be removed from your AudioProcessor and tested completely independently from the processor itself, it really shouldn’t need the processor.

You should be able to separate almost everything from the AudioProcessor itself so that hopefully all the AudioProcessor does is pull some objects together, report the name, if it supports midi, etc. and these things aren’t normally particularly useful to test.

For example do you really need to test that the name of the processor? if it failed what would it tell you? that you changed the name? well the chances are that’s exactly what you did and therefore exactly what you want. There’s not really any logic that can fail. Tests should be testing behaviour to ensure that you meant for some behaviour to change, if you add new functionality did you mean to change some existing functionality in the process.

A good way to figure out if something is likely worth testing is to think of the following three points

• Given (some state)
• when (I do something)
• Then (I expect this result)

A good way to figure out if your test is useful is to ask yourself what would I do if this ever failed, if the only real answer is change the test so it passes then it’s probably not worth testing.

In general I’m suggesting your AudioProcessor should be a very simple composition of other functions and objects, all of which are testable. This should get the level of coverage you need without over complicating anything.

If you really want to test the processor it’s certainly possible, for example

• You could define the preprocessors definitions in the console app
• You create an audio processor that takes all of the preprocessor definitions as arguments. In your console app pass these explicitly, but in the plugin pass the preprocessor definitions to the AudioProcessor in your createPluginFilter() function

Also if you do want to go this route I don’t think there should be any need for juce_audio_plugin_client as the AudioProcessor class is defined in the juce_audio_processors module. If I’m not mistaken your plugin shouldn’t depend on anything in that module.

It’s also worth noting that you can always setup tests using tools like pluginval for testing the plugin as a whole.

I hope that helps.