Crash on headless Linux (w/ xvfb or Xorg dummy)

I keep banging my head trying to get CI to work. I’m running my plugin via pluginval under Xorg or xvfb on Ubuntu 22.04. It bails out with:

X Error of failed request:  BadAtom (invalid Atom parameter)
  Major opcode of failed request:  18 (X_ChangeProperty)
  Atom id in failed request:  0x0
  Serial number of failed request:  71
  Current serial number in output stream:  82

At first I thought it was the same issue as Linux VST3 Failure with 6.1.3 and 4 but not 2 - #7 by jpo but it’s an issue with X_ChangeProperty instead of X_GetProperty, so my guess is it’s another issue with the XWindow stuff…

@attila have you seen anything like this, by any chance? I’m wondering if it’s because my plugin is opening another DocumentWindow.

I can reproduce locally on ubuntu 22.04. I’ve tried master/develop/juce7 branches, btw…it’s currently all JUCE7 (including my pluginval build).

* thread #9, name = 'pluginval', stop reason = signal SIGSEGV: invalid address (fault address: 0x10)
  * frame #0: 0x00007ffff7921f74`___pthread_mutex_lock(mutex=0x0000000000000000) at pthread_mutex_lock.c:80:23
    frame #1: 0x00007fffe7251106`XFindContext + 102
    frame #2: 0x00007ffff52d29ce`juce::getPeerFor(windowH=581) at juce_linux_XWindowSystem.cpp:1417:9
    frame #3: 0x00007ffff52df0a3`juce::XWindowSystem::windowMessageReceive(event=0x00007fffe7b75a70) at juce_linux_XWindowSystem.cpp:3851:65
    frame #4: 0x00007ffff52fd1e2`juce::XWindowSystem::initialiseXDisplay(this=0x00007fffe000e3f0, (null)=8)::$_118::operator()(int) const at juce_linux_XWindowSystem.cpp:3191:53
    frame #5: 0x00007ffff52fd061`void std::__invoke_impl<void, juce::XWindowSystem::initialiseXDisplay()::$_118&, int>((null)=__invoke_other @ 0x00007fffe7b75b68, __f=0x00007fffe000e3f0, __args=0x00007fffe7b75c04)::$_118&, int&&) at invoke.h:61:14
    frame #6: 0x00007ffff52fcff2`std::enable_if<is_invocable_r_v<void, juce::XWindowSystem::initialiseXDisplay()::$_118&, int>, void>::type std::__invoke_r<void, juce::XWindowSystem::initialiseXDisplay(__fn=0x00007fffe000e3f0, __args=0x00007fffe7b75c04)::$_118&, int>(juce::XWindowSystem::initialiseXDisplay()::$_118&, int&&) at invoke.h:111:2
    frame #7: 0x00007ffff52fcee2`std::_Function_handler<void (int), juce::XWindowSystem::initialiseXDisplay()::$_118>::_M_invoke(__functor=0x00007fffe000e3f0, __args=0x00007fffe7b75c04) at std_function.h:290:9
    frame #8: 0x00007ffff5408704`std::function<void (int)>::operator(this=0x00007fffe000e3f0, __args=8)(int) const at std_function.h:590:9
    frame #9: 0x00007ffff55ddd18`juce::LinuxEventLoop::registerFdCallback(this=0x00007fffe000e3f0)>, short)::$_3::operator()() const at juce_linux_Messaging.cpp:359:80
    frame #10: 0x00007ffff55ddcdd`void std::__invoke_impl<void, juce::LinuxEventLoop::registerFdCallback(int, std::function<void (int)>, short)::$_3&>((null)=__invoke_other @ 0x00007fffe7b75c48, __f=0x00007fffe000e3f0)>, short)::$_3&) at invoke.h:61:14
    frame #11: 0x00007ffff55ddc8d`std::enable_if<is_invocable_r_v<void, juce::LinuxEventLoop::registerFdCallback(int, std::function<void (int)>, short)::$_3&>, void>::type std::__invoke_r<void, juce::LinuxEventLoop::registerFdCallback(__fn=0x00007fffe000e3f0)>, short)::$_3&>(juce::LinuxEventLoop::registerFdCallback(int, std::function<void (int)>, short)::$_3&) at invoke.h:111:2
    frame #12: 0x00007ffff55ddb1d`std::_Function_handler<void (), juce::LinuxEventLoop::registerFdCallback(int, std::function<void (int)>, short)::$_3>::_M_invoke(__functor=0x00007fffe0002340) at std_function.h:290:9
    frame #13: 0x00007ffff50dab25`std::function<void ()>::operator(this=0x00007fffe0002340)() const at std_function.h:590:9
    frame #14: 0x00007ffff55e1a9d`juce::InternalRunLoop::dispatchPendingEvents(this=0x00007fffd8014a70) at juce_linux_Messaging.cpp:182:13
    frame #15: 0x00007ffff55d590f`juce::dispatchNextMessageOnSystemQueue(returnIfNoPendingMessages=true) at juce_linux_Messaging.cpp:342:26
    frame #16: 0x00007ffff506faa1`juce::MessageThread::start(this=0x00005555567c9c38)::'lambda'()::operator()() const at juce_LinuxMessageThread.h:70:23
    frame #17: 0x00007ffff506fa1d`void std::__invoke_impl<void, juce::MessageThread::start()::'lambda'()>((null)=__invoke_other @ 0x00007fffe7b75d68, __f=0x00005555567c9c38)::'lambda'()&&) at invoke.h:61:14
    frame #18: 0x00007ffff506f9ad`std::__invoke_result<juce::MessageThread::start()::'lambda'()>::type std::__invoke<juce::MessageThread::start(__fn=0x00005555567c9c38)::'lambda'()>(juce::MessageThread::start()::'lambda'()&&) at invoke.h:96:14
    frame #19: 0x00007ffff506f985`void std::thread::_Invoker<std::tuple<juce::MessageThread::start()::'lambda'()> >::_M_invoke<0ul>(this=0x00005555567c9c38, (null)=_Index_tuple<0UL> @ 0x00007fffe7b75da8) at std_thread.h:253:13
    frame #20: 0x00007ffff506f955`std::thread::_Invoker<std::tuple<juce::MessageThread::start()::'lambda'()> >::operator(this=0x00005555567c9c38)() at std_thread.h:260:11
    frame #21: 0x00007ffff506f8b9`std::thread::_State_impl<std::thread::_Invoker<std::tuple<juce::MessageThread::start()::'lambda'()> > >::_M_run(this=0x00005555567c9c30) at std_thread.h:211:13
    frame #22: 0x00007ffff7c952c3`___lldb_unnamed_symbol7823 + 19
    frame #23: 0x00007ffff791eb43`start_thread(arg=<unavailable>) at pthread_create.c:442:8
    frame #24: 0x00007ffff79b0a00`__clone3 at clone3.S:81

I’m not an expert on this topic, but it looks like Godot had similar issues — Atoms needed to be checked for existence before calling X_ChangeProperty.

I’m looking into pairing Xvfb/Xdummy with a window manager to see if that works around the issue…

1 Like

Be careful that you’re misinterpreting the values returned from X_ChangeProperty - use error handlers after X_* calls to validate error state, and be aware that xlib functions return 1 most of the time - this can be misinterpreted as a ‘fail because not=0’, but its there because you’re expected to use error handlers to check state:

XLib can be a constant source of frustration if you forget that you are supposed to check things almost every step of the way and not rely on function-returned values…

1 Like

Appreciate those insights Jay!

Things seem happier with a window manager (fluxbox), but still not out of the woods…It does seem that JUCE’s XLib implementation would benefit from better error handling, as you described.

Thanks for reporting. I can reproduce a very similar issue on a full desktop environment, so having a headless installation seems to be incidental. The key for me was to use --validate-in-process parameter. I’m still trying to figure out what goes wrong exactly.

1 Like

I managed to reproduce very similar crashes to this with a plugin that was creating a separate window of its own.

In my case the problem when running pluginval in command line was that the plugin would start calling Xlib functions before proper initialisation by the host. In particular it was missing the call to XInitThreads() that seemed to be the main culprit.

Xlib is initialised lazily to support use cases where X11 is not available. So I think it is the host’s responsibility to do explicit initialisation, when it knows that it’s about to do things that require it, such as opening a plugin that may use it.

A direct way of doing this would be adding the JUCE_GUI_BASICS_INCLUDE_XHEADERS=1 definition and calling juce::XWindowSystem::getInstance(). The extra includes however cause compile errors in pluginval, so a workaround is making a call that will certainly initialise Xlib.

std::make_unique<DocumentWindow> ("", Colours::black, DocumentWindow::allButtons);

Placing this line at the beginning of PluginValidatorApplication::initialise() fixed the crash for me. I hope this will work for you too.


Hey Attila, thanks so much for looking into this! You are awesome. I’m super appreciative that you went out of your way to check into pluginval, which isn’t even a part of JUCE core.

So I think it is the host’s responsibility to do explicit initialization, when it knows that it’s about to do things that require it, such as opening a plugin that may use it.

This sounds reasonable to me.

I’ll take a look to see if it resolves the issues here (I’m betting it will) and if so, I’ll submit a PR to pluginval.

Thanks again!

1 Like

Another delayed thank you to you @attila! I ended up resolving the compile errors and adding the JUCE_GUI_BASICS_INCLUDE_XHEADERS=1 — a perfect resolution and pluginval seems to finally be all the way happy on linux with Xvfb.

Esoterically, Xlib.h is doing a somewhat evil #define Status int which conflicted with a pluginval enum class.

1 Like