GUI testing in JUCE

Development
1

Hi all.

I’im the process of testing my JUCE plugin with the integrated juce UnitTest and UnitTestRunner classes. I have so far been successful in testing most of the logic and simple GUI interactions (simple “Fake” simulated clicks or scrolls or drag/drops), but I have been unsuccessful in testing any value change that interacts with the JUCE value tree.

My attepts look something like:

  • Retrieve the pointer to the component I need to test a value change on, taking the example of a ToggleButton
  • Activate the component, for example with ToggleButton::triggerClick()
  • Proceed to fail to detect any value cahange, as if the value tree listener never triggered
  • Attempt at manually triggering the value tree change with myParameters.getParameter(“toggleButtonParameter”)->setValueNotifyingHost();
  • Again failing to see anything changing

My understanding is that the message thread, which shoud be responsible of detecting these changes and notifying the listeners, is blocked by the tests themselves which run on the thread they are called from (which is in my case the main application thread). The same behaviour happens if the attempts above are carried out in a timerCallback() instead of a test: changes are not detectable until next timerCallback() call (after therefore unlocking the message thread).

Is there a native JUCE way of testing that a value tree listener has actually triggered and listened to a value, without meddling with manual threads/races/lockings or error reporting (giving up UnitTestRunner)?

Thank you in advance

Unit testing with juce::Value::Listener fails
2

Yeah, I’m facing a similar issue, have you found a solution?

3

We have an async test runner (originally written by @anthony-nicholls I believe) that runs the test suite on a separate thread, which allows us to pause that thread to wait for async operations on the message thread. I don’t think JUCE has a built-in equivalent just yet.

Some things need to be run on the message thread, so you’ll want a utility to do that from the test runner thread, pausing until it’s been run. juce::WaitableEvent is handy for this.

1 Like
4

Thank you James!

I was afraid something like this was going to be needed. I have a couple of questions though, before I go down the rabbit hole:

  • Is there any open source code that implements threaded testing? I originally thought you were talking about JIVE but it does not seem to be the case.
  • This would be partially answered by the point above, but I am afraid of polluting the codebase if Juce requires to shared-resource-thread-proofing all of the structures in order to implement threaded testing. Is this the case with WaitableEvent + timeouts or can the source code remain largely unscathed?
  • Again might be answered by the point above, does this mean that I have to give up the integrated Juce TestRunner and do the diagnostics by hand?
5

The first step of running the tests on a separate thread is really straight-forward, simply wrap the test runner in a call to juce::Thread::launch():

juce::Thread::launch([] {
    juce::UnitTestRunner runner;
    runner.runAllTests();
});

Then, in order to test the async operations, you’ll likely need a way to pause the test thread until the message thread has caught up, e.g.:

void invokeOnMessageThread (std::function<void()> function)
{
    juce::WaitableEvent event;

    juce::MessageManager::callAsync ([function, &event] {
        function();
        event.signal();
    });

    event.wait();
}

void MyTest::runTest()
{
    beginTest("something async this way comes");

    SomeObject object;
    object.triggerAsyncOperation();

    expect(object.getAsyncResult()); // This will fail because the async operation hasn't had time to run on the message thread

    invokeOnMessageThread([this] {
        expect(object.getAsyncResult()); // This should pass as it'll come after the async operation is handled by the message thread.
    });
}

No I haven’t had to use a separate thread for the JIVE test runner (yet) - I have had to do some really nasty hacky stuff to avoid that in places though.

In my experience you shouldn’t have to make any changes to your codebase to have the tests run on a background thread. You may hit a bunch of assertions requiring the message manager to be locked, but you can lock it from the tests.

2 Likes
6

Thank you again James, this has been very useful and covered all the pieces I was missing.

I only encountered minor trouble because the instances of my tests were not surviving after the Thread::launch, but it was only a matter of intantiating them earlier and outside of the routine that called Thread::lunch.

After that, your code works exactly as intended and correctly picks up async changes. Original codebase did not need modifications.

1 Like