I’m developing audio plugins using JUCE, and I’d like to do some unit testing, ideally using bandit if possible, though if it’s much simpler to get what I want using JUCE’s own unit test stuff I’m open to giving that a go.
I want to be able to do the following:
Compile and run tests (and the classes they test) quickly for fast feedback
Write and run the tests on any platform
Retain the ability to modify and re-save my project in the Projucer without having to do anything custom each time (e.g. adding new build targets in Xcode)
I’m familiar enough with C++ as a language, but don’t have all that much experience building projects that are any more than just a few small files with no library dependencies.
I can basically see two possible approaches to this:
Taking OS X as an example, I could add a new “Test” build target to Xcode which could include bandit, JUCE, etc. I’d have to do the same for VS on Windows, and add stuff to the Linux Makefile too if I wanted true cross platform development. This seems like a pain. Also, I’m not even sure if it’s possible to prevent the Projucer from overwriting that stuff every time I change a setting in there and re-save. Is that possible? If not, it more or less rules out this approach for me.
Create small test program (similar to what’s described in Unit test fixture requirements?) and write a Makefile (maybe a Makefile per OS) just to build and run my tests. Hopefully I could keep this fairly small and simple, and not have to worry about the Projucer stomping on anything. This feels to me like it should be able to offer to possibility of faster feedback times through having more control over what I need to compile, though that may be a wrong assumption.
2 seems like a winner to me, but I’m having a hard time compiling and linking the right JUCE modules with my tests and my other classes (something similar to Xcode Test Target - Linker errors, missing headers). I can post some code and my Makefile and ask for some help on that, but before I spend more time on it I’d like to understand if this is even a sensible approach. If not, why not? What’s a better approach? What am I missing?
Links to any examples of any code that’s doing focused and fast BDD using JUCE would be most welcome.
I’m not sure how wise it is to invest time chasing this. Unit testing an app with lots of DSP and UI code is a nightmare. Plus C++ doesn’t lend itself to unit testing and mocking very well compared to other languages.
The bandit example you’ve linked is basically testing properties on an object. Doesn’t offer much benefit I would say. The library itself does look nice though. I’ve used Catch before which is good too.
Sure, I get that in some ways TDD/BDD in C++ is less than ideal, and I’m sure that’s also true for some DSP and UI code. However, I still find writing unit tests to be a very fast and effective method for figuring things out, interacting with new classes, etc.
Even if I didn’t, I’d still like to understand what the best way to do this is anyway, because I’ll learn something about working with JUCE in a way that doesn’t make using Projucer a PITA later on. That alone is valuable to me.
Personally, I enjoy much the TUT unit testing framework for C++ (http://mrzechonek.github.io/tut-framework/) and have integrated it as a JUCE module (under 5.1). That is not covering GUI testing, but all libraries supporting your app. I use Projucer to build test projects on top of shared library source code, therefore developing library code directly under unit test runs. When fine, other Producer projects assemble to lib code as static or shared, and yet other build the GUI code as a thin layer over the lib. I find the whole quite productive. If any interest, I’m ready to share this TUT module.
Thanks for interest. I have uploaded a small zip archive on a web site I manage: http://www.woobe.fr/JUCE/juce_unittesting.7z that contains the JUCE module, only C++ headers actually. Just unzip it side to the other modules/juce_* and you’ll be able to see it listed in the available modules in Projucer.
Procedure:
make a new “console application” project
below is the code for the Tests_main.cpp source. Once adjusted to your convenience (e.g. export the report and integrate in nightly builds), you’ll never need to update it when you add new tests groups. In fact, each test group is a cpp source file on its own, containing numbered unit tests.
create a source cpp (no .h needed) for each test group, i.e. a common test fixture followed by diverse unit tests. See the second sample below.
when you launch the console app without arguments, all tests in all tests groups are automatically discovered and executed. A report is displayed. You can also decide to run a specific test group with “MyTestsApp test_group_name” or focus on a specific test with “MyTestsApp test_group_name test_number”. As simple as that.
The MyTestsApp main.cpp file example (to start with…):
#include "../JuceLibraryCode/JuceHeader.h"
#include <windows.h>
#include <wincon.h>
#include <xiosbase>
using namespace std;
/*------------- Unit Testing Main Program -------------*/
namespace tut
{
test_runner_singleton runner;
}
int main(int argc, const char* argv[])
{
// set the default console code page to display correctly unicode (system default is 850)
SetConsoleOutputCP(1252);
tut::console_reporter reporter;
tut::runner.get().set_callback(&reporter);
try
{
if (tut::tut_main(argc, argv))
{
if (reporter.all_ok())
{
char z; std::cin.get(z); //wait for an input before closing the console window
return 0;
}
else
{
std::cerr << "\nSome tests failed." << std::endl;
}
}
}
catch (const tut::no_such_group &ex)
{
std::cerr << "No such group: " << ex.what() << std::endl;
}
catch (const tut::no_such_test &ex)
{
std::cerr << "No such test: " << ex.what() << std::endl;
}
catch (const tut::tut_error &ex)
{
std::cout << "General error: " << ex.what() << std::endl;
}
catch (const myAPP::myException &ex) // capture app exceptions that tests would fail to trap
{
std::cout << "uncaught test error: " << ex.what() << std::endl;
}
char z; std::cin.get(z); //wait for an input before closing the console window
return 0;
}
An example test group “TestGroup01.cpp” (no header needed):
#include "../JuceLibraryCode/JuceHeader.h"
#include "../../myApp/Source/more.h" // ..... your app headers ..... libraries, etc
using namespace std;
namespace tut
{
// FIXTURE
// ==Data used by each test in this file, just declare a struct, name it as you like
struct myTestData
{
myapp::WDoc wdc; // declare ALL the objects you need here for this test group
myTestData() { // initializer for the whole struct fixture / collection
// SETUP code
wdc.doThis(); // whatever you need to prepare
}
virtual ~myTestData() { // destructor for the whole struct, release resources
// TEAR DOWN code
wdc.clear(); // ... for instance
}
};
// Test group registration
typedef test_group<myTestData> factory; // refer to your fixture
typedef factory::object object; // keep that line as is... cf below
}
namespace
{
tut::factory tf("TestGroup01-MyApp-WDoc");
// above is the name of the test group to be used as argument for executing a specific test group; it can contains spaces, you'll pass the argument within double quotes
}
namespace tut
{
/*!
Describe the first unit test here (scope, expectations, ...)
Note: the double template<> is not an error... but the whole trick!
*/
template<>
template<>
void object::test<1>() // unit tests are simply numbered 1, 2, 3 ... in sequence within a group
{
// below "ensure_..." assertion methods are part of tut namespace & source
char* buf = new char[100]; // allocate additional stuff
// check this and that...
ensure_equals( "wchar_t sizeof", sizeof(wchar_t), (unsigned)2 ); // ex: check the context
wdc.doSomething(); // simply refer to objects in your fixture...
ensure( "something that must be true", wdc.isOK() );
ensure_not( "something that must be false", wdc.checkBad() );
// some print-out's of your like...
cout<<"WDC:"<<wdc.toString()<<endl;
delete buf; // do not forget to release whatever will not vanish when going out of scope
}
/*!
scratch pad for the next test
*/
template<>
template<>
void object::test<2>()
{
// your test code here ...
}
}
