Pluginval fails on macOS if screen sharing not connected

I have a build machine with no monitor, it was working fine until recently. Now my macOS builds fail pluginval with:

Starting tests in: pluginval / Open editor whilst processing...
*** FAILED: Timeout after 30 secs

unless I have screen sharing connected. Any idea what the issue could be or how to debug?

I imagine pluginval is attempting to get the plugin under test to instantiate its GUI, and that there is an issue with WindowServer not allowing the app to launch, perhaps because, as you’re running headless, the Mac booted to the LoginWindow and there aren’t any real users logged in?

See this StackOverflow answer for more details:

The answer might be to have your system automatically log in to your test user - i.e. bypass the LoginWindow - which may work even if you don’t have a monitor physically connected.

I’ve not tried this on macOS, but for continuous integration for my Linux builds, Xvfb works well:

nohup Xvfb :1 -screen 0 1920x1080x24 &
DISPLAY=:1 ./whatever-binary

Which version, 1.0.3? Did you update recently?

Pluginval 1.0.3 seems to be happy on my plugin template headless CI here: Use WIN32 instead of MSVC · sudara/pamplejuce@8b01cbc · GitHub

But something does look fishy with the macOS pluginval CI, seems to be green but not getting very far: Update CHANGELIST.md · Tracktion/pluginval@cf70a34 · GitHub

The reason this wasn’t caught is because the AU validation action has continue-on-error: true — probably leftover when we were debugging and releasing 1.0!

/cc @dave96