Anouncing "pluginval" an open-source cross-platform plugin validation tool

When you say is “doesn’t appear to be working anymore”, what does pluginval output?

Also, what options have you go set? If you use the “–validate-in-process” option, does it crash the app?

And does VST2 work? Is it just VST3?

“pluginval v0.2.3 - JUCE v5.4.3” is the only output I get in the GUI console when starting the testing in an external process.

In-process just makes pluginval immediately disappear with my intentionally buggy plugin. I didn’t even get the Windows crash dialog.

If I remove the intentional bug in my plugin, pluginval seems to be working when doing the testing in-process.

Hmm, it sounds like it is working, but it’s crashing (as expected) before the output can be piped back to main process. That’s why you don’t get much of a log.

That shouldn’t happen though, the stack trace at the time of the crash should be sent back to the parent process.

If you try to validate other plugins does it work or do you always get a blank log?

Also, try running it from the command line with the “–validate-in-process” option. Does the log appear there or is it also truncated?

Great suggestion. I feel good.

I have an issue with a plugin with sidechain. It seems to work ok so far in several DAWs.
But when tested, I always hit this assert (EditorStressTest) in juce_AudioProcessor.cpp in setPlayConfigDetails

// if the user is using this method then they do not want any side-buses or aux outputs
success &= disableNonMainBuses();
jassert (success);

Pluginval 0.2.9, JUCE develop tip.
Any help? Can we ignore it?

Probably worth looking in to why disableNonMainBuses fails in your plugin?

I think that test sets a 0 sample rate and block size to see if your plugin can handle that before opening the editor.

I’m quite confused honestly. I don’t see why disableNonMainBuses should fail really.

So, I’ve tested the NoiseGatePluginDemo which has sidechain. The AU hits the same assert, while the VST3 hits the assert just below it:

// the processor may not support this arrangement at all
jassert (success && newNumIns == getTotalNumInputChannels() && newNumOuts == getTotalNumOutputChannels());

I don’t understand why EditorStressTest calls setPlayConfigDetails:

This is called by the processor to specify its details before being played.
Use this version of the function if you are not interested in any sidechain and/or aux buses and do not care about the layout of channels. Otherwise use setRateAndBufferSizeDetails.

I’ve tried setRateAndBufferSizeDetails and so far no asserts are hit testing both AU and VST3.

I think you’re right. setRateAndBufferSizeDetails is probably the more appropriate method here.
I’ve updated it now: Avoided issues disabling busses during an editor test · Tracktion/pluginval@82df2ea · GitHub

1 Like

Is there a way to test the plugins that have AU validation failure? I’m trying to figure out why my AudioUnits are sometimes even not recognized, or why they can’t be randomly not loaded, but I don’t see them in the plugin list :frowning:

I don’t think so I’m afraid. If they’re not recognised by auval, it usually means the AudioComponentRegistrar hasn’t registered them so JUCE won’t be able to find them either.

There was a GitHub issue about this here but I don’t think there was a definitive resolution.

It’s very unfortunate because there is no way to debug when this fails… And no indication why :frowning:

Is there a place where findPluginTypesFor is defined? One of my plugins is oddly visible by pluginval (validated properly by auval although others that are also created are not visible), but I still don’t know why it cannot open it when auval does :frowning:

Actually found that this returns in my plugin:
if (MessageManager::getInstance()->isThisTheMessageThread()
&& requiresUnblockedMessageThreadDuringCreation (desc))
return;

So as requiresUnblockedMessageThreadDuringCreation(desc) returns true, then the audio unit is not loaded…

Do you need requiresUnblockedMessageThreadDuringCreation to return true? I thought that was mainly for AUv3 support which requires it?

I don’t know, but it’s a reason why I don’t scan Audio Unit v3 in pluginval. Not sure what’s the workflow though? Is it scanned in another way after normal AUs are scanned?

I’m getting a bit confused by the problem here (sorry, it’s a symptom of doing too many things at once)…
Are you trying to run an AUv2 through pluginval or an AUv3?

If it’s an AUv2, AudioUnitPluginFormat::requiresUnblockedMessageThreadDuringCreation should return false so these lines you mentioned:

Shouldn’t return early and the plugin should be able to be loaded.

Or have I not followed correctly?

No worries, I’m confused by the way macOS works…
So for this plugin, it was an AUv3 only, so requiresUnblockedMessageThreadDuringCreation() returned true and then the plugin was not loaded at all.
To be fair, now I’m also building the AUv2, and it seems that one of the two versions is now loaded and I’m so tired by all this mess that it’s enough for me… Apple really managed to be worse than any other company in the world in messing this up.

Hello,

I am trying to run Pluginval headless on Windows & it appears to be hanging.

I am somewhere around “I am learning” status with Powershell scripting, and this seems pretty simple. That is, I am probably making a silly mistake.

The command (which works as shown without the ampersand on mac and linux) is shown below. I was getting an error message earlier that was ‘fixed’ by adding the ampersand in the front. Now it has stopped stopping and doesn’t start finishing.

& $HOME/Documents/Validators/pluginval.exe --output-dir $HOME/${HOMEDIR}/_builds/${PLUGNAME}/plug/${PLUGNAME}_artefacts/Debug/VST3 --strictness-level 5 --validate $HOME/${HOMEDIR}/_builds/${PLUGNAME}/plug/${PLUGNAME}_artefacts/Debug/VST3/${PLUGNAME}.vst3

This is what I see…

...
Time taken to run test: 7 ms

Time taken to run all tests: 2 secs
All tests completed successfully

Finished validating: C:\Users\me/Documents/SourceControl/gTest/_builds/plug-Example/plug/plug-Example_artefacts/Debug/VST3/plug-Example.vst3
ALL TESTS PASSED

Then, nothing. I was expecting to see a command prompt, but that does not happen.

Any ideas? (Thank you)

I’m not that familiar with powershell I’m afraid. In bash & is used to run a command and then return to the current shell, not waiting for it to complete. I don’t know if it does the same in powershell?

Does it do the same if you run the validation using cmd.exe?

It does work with cmd.exe. Having said that, I decided to use python rather than shell scripts as I am much more used to that (i.e. I am lazy sometimes).

I seem to be having a weird problem, though. No matter how I run it programmatically (on Windows only), validation sometimes fails. Output looks like what is below & I really don’t know how to interpret it.

pluginval v0.2.9 - JUCE v6.0.1

*** FAILED: VALIDATION CRASHED
11: BaseThreadInitThunk + 0x14
12: RtlUserThreadStart + 0x21 

Any thoughts on that one?

I have not received this ever when running pluginval manually (i.e. via the gui).

Thank you for indulging my curiosity. I feel like I should be able to automate this reliably without asking for help, but this is a bit beyond me at this point.