Amazing, thanks so much for the info. I’m excited for version 2!
TBH I think it will actually be version 1.0.0 as the current version never made it out of the 0s.
But I’m going to say it’s actually 0-indexed so v1 will be the second version 
11 Likes
Me and @sudara are pleased to announce the beta-ready state of pluginval 1.0.0: GitHub - Tracktion/pluginval: Cross platform plugin testing and validation tool
It’s been a bit of a slog but we’ve cleaned up quite a lot of the issues reported, added some features and made the whole thing generally more stable and the output more informative.
A huge thanks to @sudara on this, I probably wouldn’t have had time to do this on my own. This collaboration is a real testament to open source development!
17 Likes
Was nice to get a chance to work together!
Binaries can be downloaded here for the moment: Build · Tracktion/pluginval@092f186 · GitHub
Once Dave cuts a release, it’ll be usable in CI as before.
8 Likes
Thank you so much for putting in the work, @sudara and @dave96! PluginVal is an invaluable tool for us.
1 Like
Great work!
Now we just need all the plugin makers to use it!
2 Likes
I suspect that user error is at play here, but I’m unable to load any 3rd party AU or VST2/3 with the binary @sudara linked.
If I manually test a file it mentions code signatures being at fault, any idea what I should do to solve this issue?
Started validating: /Library/Audio/Plug-Ins/VST/Serum.vst
Started validating: /Library/Audio/Plug-Ins/VST/Serum.vst
Random seed: 0x1f0a933
Validation started: 12 Jul 2022 11:57:07am
Strictness level: 5
-----------------------------------------------------------------
Starting tests in: pluginval / Scan for plugins located in: /Library/Audio/Plug-Ins/VST/Serum.vst...
Attempting to load VST: /Library/Audio/Plug-Ins/VST/Serum.vst
2022-07-12 11:57:07.565 pluginval[20019:1201442] Error loading /Library/Audio/Plug-Ins/VST/Serum.vst/Contents/MacOS/Serum: dlopen(/Library/Audio/Plug-Ins/VST/Serum.vst/Contents/MacOS/Serum, 262): no suitable image found. Did find:
/Library/Audio/Plug-Ins/VST/Serum.vst/Contents/MacOS/Serum: code signature in (/Library/Audio/Plug-Ins/VST/Serum.vst/Contents/MacOS/Serum) not valid for use in process using Library Validation: mapping process and mapped file (non-platform) have different Team IDs
Num plugins found: 0
!!! Test 1 failed: No types found. This usually means the plugin binary is missing or damaged, an incompatible format or that it is an AU that isn't found by macOS so can't be created.
FAILED!! 1 test failed, out of a total of 1
*** FAILED
Finished validating: /Library/Audio/Plug-Ins/VST/Serum.vst
*** FAILED WITH EXIT CODE: 1
Finished batch validation
edit:
given this:
not valid for use in process using Library Validation
I tried switching to “Validate in process”, unfortunately this only changed the output slightly:
Random seed: 0x5399cc7
Validation started: 12 Jul 2022 12:13:25pm
Strictness level: 5
-----------------------------------------------------------------
Starting tests in: pluginval / Scan for plugins located in: /Library/Audio/Plug-Ins/VST/Serum.vst...
Started validating: /Library/Audio/Plug-Ins/VST/Serum.vst
Num plugins found: 0
!!! Test 1 failed: No types found. This usually means the plugin binary is missing or damaged, an incompatible format or that it is an AU that isn't found by macOS so can't be created.
FAILED!! 1 test failed, out of a total of 1
Finished validating: /Library/Audio/Plug-Ins/VST/Serum.vst
*** FAILED WITH EXIT CODE: 1
Finished batch validation
I still suspect that it’s something dumb that I have done, or some annoying macOS issue (I replaced an older version of pluginval with the new binary in my /Applications folder).
Thanks for the work though, hopefully will get to the bottom of this problem.
It’s not you! I swept the entitlements under the rug instead of you know, actually code signing with them. New build will be ready in a few min…
1 Like
Ok, bit of a delay due to some CI wrastlin’, but here’s an actual pre-release (with entitlements): Release v1.0.0-beta4 · Tracktion/pluginval · GitHub
It can be run in CI same as before. To try out the beta in CI, use the following urls:
https://github.com/Tracktion/pluginval/releases/download/v1.0.0-beta4/pluginval_Linux.zip
https://github.com/Tracktion/pluginval/releases/download/v1.0.0-beta4/pluginval_macOS.zip
https://github.com/Tracktion/pluginval/releases/download/v1.0.0-beta4/pluginval_Windows.zip
You can see an example of a GitHub Actions config here which is currently green on a trivial JUCE vst3 on macOS, Windows and Linux (all running at strictness level 10).
Note that for Linux you’ll need to run a display emulator like Xvfb and set $DISPLAY or run with --skip-gui-tests.
Also, every commit to pluginval itself now builds several JUCE example plugins and validates them on all 3 platforms — this will do a lot to preemptively catch issues on all platforms and ensure stability going forward.
8 Likes
Great work on the new version, and thanks for the bug fixes! My previously mentioned issue is now solved.
One small thing I noticed with the new version is that if any other arguments are specified after --validate <pluginPath>, it appears to interpret the remaining arguments as plugin names.
For example, the invocation
pluginval --validate somePlugin.vst3 --randomise
gives the output:
Started validating: --randomise
Random seed: 0x71d74ff
Validation started: 14 Jul 2022 6:33:37am
Strictness level: 10
-----------------------------------------------------------------
Starting tests in: pluginval / Scan for plugins located in: --randomise...
Num plugins found: 0
!!! Test 1 failed: No types found. This usually means the plugin binary is missing or damaged, an incompatible format or that it is an AU that isn't found by macOS so can't be created.
FAILED!! 1 test failed, out of a total of 1
*** FAILED
This got me thinking that maybe it would be nice to lower the barrier to entry. Give people a taste, get em hooked on the good stuff—er, best practices.
I tried to cook this up in 24h (lol) but ended up in a few days of cross-platform hell. But it SHOULD work for VST3/AU on all platforms with the one caveat that macOS has to be universal or x86_64 (not arm-only, as the runners aren’t arm yet).
Results example:
Let me know if anyone spots anything unexpected (yeah, those urls are ridiculous!).
11 Likes
That’s pretty slick. What are you using for the runners – is this going to cost you a bunch of money if it gets popular?
1 Like
Thanks! Runners are Github workflows for now, as I had that all setup already. Let’s see how things go, maybe a sponsor will be in order for the long term…
1 Like
thanks, that’ super cool.
I get “ERROR: failed to download log file” on a working plugin.
1 Like
Thanks! Looking into it…
I made a new thread for validatemyplugin dot com questions/issues (so this one can stick to pluginval proper).
Thanks to everyone for kicking the tires. pluginval 1.0 came out of beta this week and Dave cut 1.0.1 to support older versions of Ubuntu:
7 Likes
Hi guys,
I noticed you totally removed the projucer project for pluginval, going exclusively CMake. Unfortunately I don’t use CMake and I always built pluginval with the JUCE version we use on production. Any chance to have that back or I’ll have to do that from scratch?
Thanks!
Luca
It was getting too difficult to maintain both and cmake really is a better choice these days. It’s more flexible and works better with other tooling.
I’d strongly recommend just installing cmake, it’s then just a single command line argument to build the IDE files which is actually easier than opening the PJ project and saving it.
Or you can directly grab the pre-built binaries if you don’t need to build it from source.
3 Likes