Thanks for the interesting tool! My own plugin passed 3 times successfully at level 10.
I however found a plugin from another developer that crashes. That however didn’t cause the plugin to be reported as failed, is that expected behavior from the validator? (I suspect the problem happens at plugin destruction time…)
Hmm, it should get reported as failed, you should get a log message with something like
"ERROR! Plugin crashed"
Is this repeatable? Can you let me know the plugin, platform and format so I can check?
Just for info really… but we had one that crashed on destruction due to removing change listeners in the destructor of the processor, and this was being called on a thread other than the message thread.
Can anyone please test this plug-in on Windows 10 on their system (free demo):
It fails to scan using the JUCE Host… but does work in Tracktion Waveform.
When I drag the .vst onto pluginVal it shows:
Exception thrown at 0x00007FFA31CABD29 (bx_console SSL 4000 E.dll) in pluginval.exe: 0xC0000005: Access violation reading location 0x00000000101D012C. occurred
at line 2136 in juce_VSTPluginFormat.cpp
{
JUCE_VST_WRAPPER_INVOKE_MAIN
}
Thanks,
Rail
First of all, thanks a lot @dave96 I’ve been using this A LOT! It’s now integrated (along with auvaltool) in our automated build process.
I have a question and maybe some ideas for additional tests.
In AudioProcessingTest, should the checks for NaNs, Infs and Subnormals be done for each buffer processed?
Some ideas for additional tests:
bpm, timeSigNumerator and timeSigDenominator. It would be nice to check against those.AudioProcessingTest or AutomationTest. Unless I’m missing something, I see that the AudioProcessingTest processes 10 buffers for each block size and sample rate, and AutomationTest only one buffer? I don’t think this is enough for some of the very nasty bugs, and I don’t mind longer tests as an option, just to be safe.Thanks for using! I just wanted to reply quickly to let you know I’m at CppCon atm but will take a proper look at this next week.
Absolutely, take your time 
If I drag and drop my audioUnit to PluginVal from ~/Library/Audio/Plug-Ins/Components/
it won’t work, the plugin is not added to the list.
But if I copy it to : /Library/Audio/Plug-Ins/Components/ and drag it from there to pluginVal, then everything is fine.
Anyone else noticed that?
I kind of remember there were some auval issues on high-sierra, and that users had to reboot for auval/logic to see the new plugins. I guess that’s related?
I seem to remember this being a quirk of macOS and it not being able to find the AU Component if it’s not in the system folder (we have to have it there on our CI for auval to find it).
You could dig down into the JUCE plugin scanning code, there’s an AU code path that tried to look in the dropped file for a bundle ID but if the AUComponent library fails to return a valid component for it there’s not a lot we can do… Maybe file a bug with Apple?
Sorry for the delay, just got back and am still catching up on things.
Yes, I’ve added this to the list of tests to add. I don’t have any setting the AudioPlayHead yet so will make sure I add these soon.
Yeah, at the moment I just use a small number for buffers for quick testing, this number will vary between tests. For the AutomationTest, the main thing it’s testing is that parameter values can be set within sub-blocks so the number of setParameter calls will vary with the block size.
For improved flexibility I have some plans to add more user-definable tests (with custom data and possibly Javascript based tests) but this is a little out of scope for the validation nature of the current version.
Yep, this will hopefully be coming very soon.
N.B. my ADC talk is on pluginval and the things it tests for in general so hopefully I’ll get to some of these more testing based features before then!
Awesome, thanks a lot! Can’t wait
Hi all, as you may or may not know I’ll be giving a talk on pluginval, testing plugins and host <-> plugin issues at ADC this year.
I’d like to get an idea of how many people are using pluginval for this so if you’re finding it useful ping me a message here or a PM. Also let me know if you’d like me to feature your company/logo on a slide.
Cheers!
Just a quick update to let you know that out “Verified by pluginval” page is now live.
More info here:
I just pulled the latest commits (fcb23eb ) and the notes say 0.1.5 but the plist says 0.1.3 and in the ValidatorSlaveProcess ctor there’s a call to initialiseCrashHandler() but I get a link error in Xcode and I don’t find that method/function anywhere in the code.
Cheers,
Rail
Hmm, it’s passing on Travis…
initialiseCrashHandler is in pluginval/Source/CrashHandler.cpp. Maybe you need to re-generate the project with PJ via the install/mac_build script?
I just dragged CrashHandler.cpp/.h into the project and changed the version to 0.1.5 and it compiles. Any reason you don’t have the solution/project (build folder) in the repos.?
Cheers,
Rail
Sounds like you need to do a git clean -dfx or run the build script I mentioned to get things up to date.
The IDE files are really just clutter and it’s difficult to match exact version etc. with everyone’s machines. For years we’ve been taking the minimal/generate locally approach by using the PJ to generate the projects.
At Tracktion, we always ended up with local IDE files being changed by the IDEs themselves (i.e. they didn’t match exactly what the PJ produces) after being opened then these would get checked-in and then there would be local conflicts and you wouldn’t know who’s was right etc.
Using the .jucer file as master is just much cleaner.
I think this is generally the more accepted way of doing things these days and more closely follows the CMake paradigm which is getting more and more popular these days…
BTW - With the same source… VST2 passes at level 10, and generates:
Starting test: pluginval / Listing available buses…
Inputs:
Named layouts: None
Discrete layouts: None
Outputs:
Named layouts: Stereo, Stereo, Stereo, Stereo, Stereo, Stereo, Stereo, Stereo, Stereo, Stereo, Stereo, Stereo, Stereo, Stereo, Stereo, Stereo
Discrete layouts: None
Main bus num input channels: 0
Main bus num output channels: 2
All tests completed successfully
but VST3 fails at level 5 with:
Starting test: pluginval / Listing available buses…
Inputs:
Named layouts: None
Discrete layouts: None
*** FAILED: Timeout after 30 secs
I have no issues running my plug-in in any host.
My main processor ctor is:
MyAudioProcessor::MyAudioProcessor() : AudioProcessor (BusesProperties()
.withOutput ("Out 1-2", AudioChannelSet::stereo(), true)
.withOutput ("Out 3-4", AudioChannelSet::stereo(), false)
.withOutput ("Out 5-6", AudioChannelSet::stereo(), false)
.withOutput ("Out 7-8", AudioChannelSet::stereo(), false)
.withOutput ("Out 9-10", AudioChannelSet::stereo(), false)
.withOutput ("Out 11-12", AudioChannelSet::stereo(), false)
.withOutput ("Out 13-14", AudioChannelSet::stereo(), false)
.withOutput ("Out 15-16", AudioChannelSet::stereo(), false)
.withOutput ("Out 17-18", AudioChannelSet::stereo(), false)
.withOutput ("Out 19-20", AudioChannelSet::stereo(), false)
.withOutput ("Out 21-22", AudioChannelSet::stereo(), false)
.withOutput ("Out 23-24", AudioChannelSet::stereo(), false)
.withOutput ("Out 25-26", AudioChannelSet::stereo(), false)
.withOutput ("Out 27-28", AudioChannelSet::stereo(), false)
.withOutput ("Out 29-30", AudioChannelSet::stereo(), false)
.withOutput ("Out 31-32", AudioChannelSet::stereo(), false)
),
:
Rail
*** FAILED: Timeout after 30 secs is the key here.
Can you run it in debug mode with “Validate in process” on (so you can debug it).
Run the validation and it will pause presumably after starting the Listing available buses test. Can you pause the debugger and see where it is and why it’s taking more than 30 seconds to get out of there?
Seems protection related… my Release (unprotected) build passes all tests @ 10. I’ll re-analyze the startup code for VST3.
Thanks,
Rail
