# Automated DSP Testing **URL:** \ **Category:** Audio Plugins\ **Created:** [June 20, 2017, 9:29pm UTC](https://forum.juce.com/t/automated-dsp-testing/22998 "2017-06-20T21:29:44Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![tadnicol](https://sea2.discourse-cdn.com/flex026/user_avatar/forum.juce.com/tadnicol/32/5141_2.png) [@tadnicol](https://forum.juce.com/u/tadnicol)\ **Post date:** [June 20, 2017, 9:29pm UTC](https://forum.juce.com/t/automated-dsp-testing/22998/1 "2017-06-20T21:29:44Z")
Hi JUCE community 🙂 I’m working on some plugins & native apps that need to attain & retain a high standard of DSP processing. Thus I need to design a system that can load VST/AU/AAX plugins _and_ C++ functions/classes on macOS/Windows and run a bunch of tests (which are defined by the math wizard / DSP guru in my company). This system will be integrated into my build pipeline so the tests can be run easily after a new build is kicked off. Some of the requirements I’m looking in such a system: • Command line compatible (ie a GUI and human-clicking is not required) • Spectral Feature processing/analyzing • General DSP metering algorithms (calculating THD, phase response, frequency response, various amplitude/loudness measurements, etc.) • Framework for organizing/running tests. Must be able to load/handle audio file formats • Plotting/graphing capabilities for interfacing with humans when needed • Something that 1 full-time developer can implement/handle Right now I’m looking at using [Boost.Python](http://www.boost.org/doc/libs/1_49_0/libs/python/doc/) to write a Python module that would wrap JUCE’s `AudioPluginInstance` class & plugin loading code so that I could load all the different formats of our plugins and tweak their parameters and test different presets, etc. I’m thinking of using Python to write the tests because: • Quick implementation • Lots of great existing libraries: [SciPy](https://www.scipy.org/scipylib/index.html), [MatPlotLib](http://matplotlib.org/), [NumPy](http://www.numpy.org/), etc. • Plotting capabilities a la MatPlotLib (useful for interfacing with humans, eg say a test fails and I want to report to our DSP guru) • Great open-source community with lots of other modules available (maybe making use of some of the machine learning libs would be useful in the near future 😮) I was wondering if anybody else has any stories/tips they could share if they’ve tried to build a similar testing system. Has anybody had success with existing audio plugin wrapper Python libs? Does anybody know of some amazing open source C++ libraries that could offer similar functionality? If anybody else is interested in the JUCE `AudioPluginInstance` wrapper for Python I mentioned above, I’d be very down to collaborate on an open source project. ---
**Author:** ![matthieu-brucher](https://avatars.discourse-cdn.com/v4/letter/m/a5b964/32.png) [@matthieu-brucher](https://forum.juce.com/u/matthieu-brucher)\ **Post date:** [June 20, 2017, 11:02pm UTC](https://forum.juce.com/t/automated-dsp-testing/22998/2 "2017-06-20T23:02:11Z")
Your almost on track, but instead of Boost.Python, use pybind11, far better and up to date with C++ standards. This is what I’m now using for Audio ToolKit which seems to be close to what you want to do. Still, I would advise to wrap the components inside of your instances, not just the instances themselves. This would only be functional tests when you also require unit tests. Testing DSP code with a high standard doesn’t require VST/AU/AAX testing, they are orthogonal. Especially given what you want to test (I’m testing the DSP pipelines built in Python even if they are also in C++ in my plugins). ---
**Author:** ![tadnicol](https://sea2.discourse-cdn.com/flex026/user_avatar/forum.juce.com/tadnicol/32/5141_2.png) [@tadnicol](https://forum.juce.com/u/tadnicol)\ **Post date:** [June 21, 2017, 3:08pm UTC](https://forum.juce.com/t/automated-dsp-testing/22998/3 "2017-06-21T15:08:51Z")
> [@Matthieu\_Brucher](#): > > Your almost on track, but instead of Boost.Python, use pybind11, far better and up to date with C++ standards. NIce! Thanks Matthieu! It looks like a much leaner alternative as well which is a big plus 🙂 I’m planning on doing both small units tests to individual functions as well as testing the overall product that the end user interfaces with (ie testing the DSP modules connected together). So having elegant C++/Python interoperability will be really nice 🙂 ---
**Author:** ![matthieu-brucher](https://avatars.discourse-cdn.com/v4/letter/m/a5b964/32.png) [@matthieu-brucher](https://forum.juce.com/u/matthieu-brucher)\ **Post date:** [June 21, 2017, 3:24pm UTC](https://forum.juce.com/t/automated-dsp-testing/22998/4 "2017-06-21T15:24:56Z")
I agree. I always try to design my plugins with my Python layers, it’s really easy to check results, display them, profile independently of the host… And yes, leaner, you don’t need to precompile something. The inconvenience is when you have inheritance across Python modules, but still manageable. ---
**Author:** ![SimonZimmer](https://avatars.discourse-cdn.com/v4/letter/s/e95f7d/32.png) [@SimonZimmer](https://forum.juce.com/u/SimonZimmer)\ **Post date:** [April 20, 2019, 9:21am UTC](https://forum.juce.com/t/automated-dsp-testing/22998/5 "2019-04-20T09:21:47Z")
Hey 👋 I’m also trying to automate my DSP testing with python/juce and I’m currently using pybind11, building my python module with cmake. This works fine when using plain C++. I’m running into issues though when I try to introduce juce into the process and I’m wondering weather some of you guys got it going in the meantime and could help me out? I’m including juce with the following CMake lines: ``` target_include_directories (my_module PUBLIC "../../JuceLibraryCode" "../../../JUCE/modules" ) ``` Which does compile my module in release mode. Unfortunately, when I’m trying to include my module from a python script I get the following error: `__ZN4juce78this_will_fail_to_link_if_some_of_your_compile_units_are_built_in_release_modeC1Ev` Curiously, when I’m compiling in debug mode, I get: `__ZN4juce76this_will_fail_to_link_if_some_of_your_compile_units_are_built_in_debug_modeC1Ev` Heres my CMake.txt in debug mode: ``` cmake_minimum_required (VERSION 3.0.0) set(CMAKE_VERBOSE_MAKEFILE ON) project(my_module) add_subdirectory(pybind11) pybind11_add_module(my_module ../../Source/Main.cpp) target_include_directories (my_module PUBLIC "../../JuceLibraryCode" "../../../JUCE/modules" ) target_compile_definitions(my_module PRIVATE "-D_NDEBUG=1" "-DNDEBUG=1" ) ``` and my Main.cpp: ``` #include "../JuceLibraryCode/JuceHeader.h" #include "../Builds/CLion/pybind11/include/pybind11/pybind11.h" PYBIND11_MODULE(my_module, m) { m.def("add", [](int a, int b) {return a + b;}); } int main(int argc, char* argv[]) { return 0; } ``` Any help would be very much appreciated! Best, Simon ---
**Author:** ![mcmartin](https://sea2.discourse-cdn.com/flex026/user_avatar/forum.juce.com/mcmartin/32/7432_2.png) [@mcmartin](https://forum.juce.com/u/mcmartin)\ **Post date:** [April 20, 2019, 9:57am UTC](https://forum.juce.com/t/automated-dsp-testing/22998/6 "2019-04-20T09:57:14Z")
Hello 👋 That’s very courageous of you to try to integrate JUCE into a CMake project! I have quite some experience with it, so I think I can help. First of all, where is the JUCE code being compiled or linked into your CMake project? The errors you are getting look like linker errors, which means that there are no objects files or static libraries that contain the compiled JUCE code. Then `NDEBUG` and `_NDEBUG` should be defined only when the configuration is _not_ Debug. On the other hand, `DEBUG` and `_DEBUG` should be defined only in Debug configuration. You can achieve that using generator expressions: ```auto target_compile_definitions(my_module PUBLIC $<$:_DEBUG> $<$:DEBUG> $<$>:_NDEBUG> $<$>:NDEBUG> ) ``` I hope this helps. Feel free to ask me more questions (here or by direct message). ---
**Author:** ![SimonZimmer](https://avatars.discourse-cdn.com/v4/letter/s/e95f7d/32.png) [@SimonZimmer](https://forum.juce.com/u/SimonZimmer)\ **Post date:** [April 20, 2019, 11:27am UTC](https://forum.juce.com/t/automated-dsp-testing/22998/7 "2019-04-20T11:27:09Z")
Hey McMartin, Thank you very much for replying this quickly! I’m happy to have reached you, as I’m quite new to CMake. I’m answering here if that’s ok, maybe someone will profit from this conversation being public one day 🙂 > First of all, where is the JUCE code being compiled or linked into your CMake project? Yes I should have documented that. I launched the Projucer once and let it generate the CMake.txt file for me using the CLion beta support. In the settings of the projucer I specified to build a dynamic library and linked my Python installation. At first CMake didn’t find a whole lot of Python Symbols which are included from pybind11. I could resolve this by adding the `undefined dynamic_lookup` flag to `set (CMAKE_CXX_FLAGS)`. After that Cmake could successfully build the dynamic library with the pybind11 include in my Main.cpp. I realize now that I have to somehow link the `my_module` target to the compiled juce modules? Honestly, I don’t know how to do that exactly. thanks for the automatic debug/release expression, which is already very helpful! Heres my project structure, with the compiled juce modules and Main file: ![cmake_juce_pybind11](https://us1.discourse-cdn.com/flex026/uploads/juce/original/2X/f/f3f4571ff0e21255a8fb272c0c03f6f97c2144bc.jpeg) ---
**Author:** ![SimonZimmer](https://avatars.discourse-cdn.com/v4/letter/s/e95f7d/32.png) [@SimonZimmer](https://forum.juce.com/u/SimonZimmer)\ **Post date:** [May 10, 2019, 9:51am UTC](https://forum.juce.com/t/automated-dsp-testing/22998/8 "2019-05-10T09:51:57Z")
I ended up going back to a regular Xcode project and integrated pybind11, which was a pretty straightforward header-only library include and works great! As it’s only a small project it wasn’t that important to bring CMake into the picture. Thank you very much for your help! cheers, Simon ---
**Author:** ![Ayra](https://sea2.discourse-cdn.com/flex026/user_avatar/forum.juce.com/ayra/32/5775_2.png) [@Ayra](https://forum.juce.com/u/Ayra)\ **Post date:** [December 8, 2021, 8:40pm UTC](https://forum.juce.com/t/automated-dsp-testing/22998/9 "2021-12-08T20:40:50Z")
I, I know that it’s not the thread to put my question, but reading I think you are the most suitable people for it. How to integrate pybind11 in projucer? I have a juce project that I’m develop on xcode and I really need ti use some python libraries to achieve my aims I wrote the path of my pybind11 on extra framework path of projucer but when in code i write #include \ (even with quotes instead “\<” and “\>”) it tells me that it doesn’t know it. Really thank you in advance and sorry for the trouble, but this thing is gonna make me mad 🙏 ---
**Author:** ![mcmartin](https://sea2.discourse-cdn.com/flex026/user_avatar/forum.juce.com/mcmartin/32/7432_2.png) [@mcmartin](https://forum.juce.com/u/mcmartin)\ **Post date:** [December 8, 2021, 9:31pm UTC](https://forum.juce.com/t/automated-dsp-testing/22998/10 "2021-12-08T21:31:55Z")
pybind11 is header-only, it is not a macOS framework (which have the extension `.framework`). This means only the compiler needs to know where to _search_ the pybind11 _headers_ to resolve the `#include` statements. As highlighted in the previous sentence, this is done using the “Header Search Paths” fields, either the one at the project level, or the ones at the configuration level. There is however a small gotcha, these “Header Search Paths” are not relative to the `.jucer` project file. They are instead relative to the generated build system files (like the Xcode project or the Visual Studio solution). I hope this helps!