Hi,
I’m new to Juce and wonder how the development cycle of a audio plugins looks like.
After a code change which you want to test, do you have to load the newly compiled plugin into some host each time to test it?
Or is there a faster way to speed up development? Because that would make the iteration cycle very slow.
That’s pretty much it. You can configure your IDE of choice so that it runs the DAW and then just have your DAW open with a default project containing your plugin on a track. With this you can run and step debug your plugin as you would any other program. You’ll also get debug output from the DAW/your plugin in your IDE.
FWIW, I use Reaper for this purpose. It’s the fastest DAW I’ve used for boot up.
The other plus for debugging this way is that you can configure your DAW project to have spectrum analysers and oscilloscopes etc already loaded along with any other plugins useful for dev.
A bit of a pain but it’s the way to go. If you’re not testing plugin related features, but purely GUI or DSP stuff, you can also consider building the plugin as a standalone app, that makes the process a lot faster. But since your users will be using it inside a DAW, it can’t hurt to test in that environment a lot.
JUCE comes with a very simple host called AudioPluginHost that starts up very fast that is useful for debugging.
The standalone target is faster and more convenient for debugging for most cases, turnaround times significantly reduced.
often times you get away with just loading the standalone version of the plugin build to test your features. it builds the plugin into sort of a little pseudo host in order to act standalone. but that host has no sidechain inputs and no concept of a playhead (bpm, ppq etc) and ofc doesn’t let you come up with effect chains for different test scenarios of your plugin.
in such cases you can run the juce standalone host as a debugger. but it also has no playhead, so then you need a DAW.
generally i’d say using a DAW is way less of a hassle than the standalone debugger, because you can freely create any scenario, have all the features of your plugin available and while you’re at it you can already find some bugs that only expose themselves to you if you run it in hosts that behave a little different than what you’d expect.
DAWs that can be run as a debugger, like reaper or cubase, are useful for finding bugs that already happen at initialization with the breakpoint debugger. but usually i use bitwig as a test host. due to its plugin sandboxing system you can almost hotswap builds and speed up your casual troubleshooting workflows
This is the indie way.
But there are also automated testings, but I don’t know of a off-the-shelf solution.
For automated fuzz tests / smoke tests you run through tracktion pluginval, Apples auval, Steinbergs VST3Validator and AVID’s DSH tool. Pick the ones that apply for you. pluginval is highly recommended.
You can do A/B testing, which requires a scriptable host and you compare the rendered output with the original.
This is a good talk about testing (just because I happened to attend there and found it interesting, there might be more talks to watch):
It starts with the A/B cancellation testing, which I find well applicable to the topic.
Thanks for the answers.
What is not clear to me is in which of these scenarios will we be able to debug the plugin in Visual Studio debugger. Thus setting breakpoints and stepping through the code if needed.
The debugger can only be attached to a process. What you do is you attach the debugger to the host process or the validator. This allows you to add breakpoints in your plugin.
Not all hosts allow that though.
I definitely don’t load the plugin in a DAW and manually test after every code change. That would take way too much time. I highly recommend having as many unit tests as possible. Then the workflow looks like: make a change, then run all the unit tests, make another change, run tests again, then once you’re at a new stable version, then you manually test in a DAW.
Others have mentioned pluginval, I recommend registering pluginval to run as a test case with your test harness (CTest or other). Unit tests for GUI are more difficult to write, though it can be done; I mostly write unit tests for DSP code.
The one I gave in the first response post. Your IDE runs the DAW. Have a default project that loads with your plugin on a track. The IDE debug session will bind and you can debug step with breakpoints in your plugin. Inspect variables, step into/over and so on.
I’ve got that set up with Reaper and VSCode and can go from hitting run to my first breakpoint in just a few seconds. For some projects I’ve got the same with XCode and it’ll be the same deal with Visual Studio.
Not sure if I understand this.
Do you mean that after compiling the plugin the IDE starts some DAW that automatically loads the plugin? And then you can use breakpoints in your plugin and step through the code?
(I use Visual Studio)
@benvining is an expert on continuous integration. A build machine that executes as much of the testing automatically, so you can concentrate on coding and get a heads up whenever something broke.
Ben helped us adding more tests and automate the build chain, which involves much more, like code signing, notarisation, said pluginval and creating installers.We are pretty happy we did that.
But that is maybe step two…
It’s not “some DAW”, it’s one you have to set up yourself. So if you have something like Reaper, Cubase or Ableton Live installed, you’d set that up in the Visual Studio project/debugging properties.
Juce itself comes with the AudioPluginHost (that you have to build from source) which is a lightweight simple host suitable for some basic tests.
When I develop an audio plugin, I use the JUCE standalone wrapper a lot and unit tests of course. This is a great approach for the parts of the plugin that don’t really need to be tested inside of a DAW. For instance, GUI development can be done perfectly in the standalone wrapper. (of course I will test the GUI in a DAW once in a while, but not after every change)
However, to test interactions with DAW’s (such as automation, offline rendering, bus layouts, etc.), manually testing in DAW’s is necessary if you want to be sure the plugin functions properly. When running your plugin in a DAW, make sure to attach your IDE to the DAW so that you can debug things with breakpoints and asserts.
In Visual Studio, under the project properties 'Debug" tab, you can set which DAW to load (when you hit F5 ‘Start Debugging’). You should not be having to waste time manually starting a DAW and loading a project, it should all be triggered by a single keypress. Otherwise you are going to burn a lot of time and money doing it manually, again and again.
Small tip when using Reaper: disable the splash screen in the preferences and save between 1 and 2 seconds on each startup.
