Crash on headless Linux (w/ xvfb or Xorg dummy)

Linux
1

I keep banging my head trying to get CI to work. I’m running my plugin via pluginval under Xorg or xvfb on Ubuntu 22.04. It bails out with:

X Error of failed request:  BadAtom (invalid Atom parameter)
  Major opcode of failed request:  18 (X_ChangeProperty)
  Atom id in failed request:  0x0
  Serial number of failed request:  71
  Current serial number in output stream:  82

At first I thought it was the same issue as Linux VST3 Failure with 6.1.3 and 4 but not 2 - #7 by jpo but it’s an issue with X_ChangeProperty instead of X_GetProperty, so my guess is it’s another issue with the XWindow stuff…

@attila have you seen anything like this, by any chance? I’m wondering if it’s because my plugin is opening another DocumentWindow.

2

I can reproduce locally on ubuntu 22.04. I’ve tried master/develop/juce7 branches, btw…it’s currently all JUCE7 (including my pluginval build).

* thread #9, name = 'pluginval', stop reason = signal SIGSEGV: invalid address (fault address: 0x10)
  * frame #0: 0x00007ffff7921f74 libc.so.6`___pthread_mutex_lock(mutex=0x0000000000000000) at pthread_mutex_lock.c:80:23
    frame #1: 0x00007fffe7251106 libX11.so.6`XFindContext + 102
    frame #2: 0x00007ffff52d29ce MyPlugin.so`juce::getPeerFor(windowH=581) at juce_linux_XWindowSystem.cpp:1417:9
    frame #3: 0x00007ffff52df0a3 MyPlugin.so`juce::XWindowSystem::windowMessageReceive(event=0x00007fffe7b75a70) at juce_linux_XWindowSystem.cpp:3851:65
    frame #4: 0x00007ffff52fd1e2 MyPlugin.so`juce::XWindowSystem::initialiseXDisplay(this=0x00007fffe000e3f0, (null)=8)::$_118::operator()(int) const at juce_linux_XWindowSystem.cpp:3191:53
    frame #5: 0x00007ffff52fd061 MyPlugin.so`void std::__invoke_impl<void, juce::XWindowSystem::initialiseXDisplay()::$_118&, int>((null)=__invoke_other @ 0x00007fffe7b75b68, __f=0x00007fffe000e3f0, __args=0x00007fffe7b75c04)::$_118&, int&&) at invoke.h:61:14
    frame #6: 0x00007ffff52fcff2 MyPlugin.so`std::enable_if<is_invocable_r_v<void, juce::XWindowSystem::initialiseXDisplay()::$_118&, int>, void>::type std::__invoke_r<void, juce::XWindowSystem::initialiseXDisplay(__fn=0x00007fffe000e3f0, __args=0x00007fffe7b75c04)::$_118&, int>(juce::XWindowSystem::initialiseXDisplay()::$_118&, int&&) at invoke.h:111:2
    frame #7: 0x00007ffff52fcee2 MyPlugin.so`std::_Function_handler<void (int), juce::XWindowSystem::initialiseXDisplay()::$_118>::_M_invoke(__functor=0x00007fffe000e3f0, __args=0x00007fffe7b75c04) at std_function.h:290:9
    frame #8: 0x00007ffff5408704 MyPlugin.so`std::function<void (int)>::operator(this=0x00007fffe000e3f0, __args=8)(int) const at std_function.h:590:9
    frame #9: 0x00007ffff55ddd18 MyPlugin.so`juce::LinuxEventLoop::registerFdCallback(this=0x00007fffe000e3f0)>, short)::$_3::operator()() const at juce_linux_Messaging.cpp:359:80
    frame #10: 0x00007ffff55ddcdd MyPlugin.so`void std::__invoke_impl<void, juce::LinuxEventLoop::registerFdCallback(int, std::function<void (int)>, short)::$_3&>((null)=__invoke_other @ 0x00007fffe7b75c48, __f=0x00007fffe000e3f0)>, short)::$_3&) at invoke.h:61:14
    frame #11: 0x00007ffff55ddc8d MyPlugin.so`std::enable_if<is_invocable_r_v<void, juce::LinuxEventLoop::registerFdCallback(int, std::function<void (int)>, short)::$_3&>, void>::type std::__invoke_r<void, juce::LinuxEventLoop::registerFdCallback(__fn=0x00007fffe000e3f0)>, short)::$_3&>(juce::LinuxEventLoop::registerFdCallback(int, std::function<void (int)>, short)::$_3&) at invoke.h:111:2
    frame #12: 0x00007ffff55ddb1d MyPlugin.so`std::_Function_handler<void (), juce::LinuxEventLoop::registerFdCallback(int, std::function<void (int)>, short)::$_3>::_M_invoke(__functor=0x00007fffe0002340) at std_function.h:290:9
    frame #13: 0x00007ffff50dab25 MyPlugin.so`std::function<void ()>::operator(this=0x00007fffe0002340)() const at std_function.h:590:9
    frame #14: 0x00007ffff55e1a9d MyPlugin.so`juce::InternalRunLoop::dispatchPendingEvents(this=0x00007fffd8014a70) at juce_linux_Messaging.cpp:182:13
    frame #15: 0x00007ffff55d590f MyPlugin.so`juce::dispatchNextMessageOnSystemQueue(returnIfNoPendingMessages=true) at juce_linux_Messaging.cpp:342:26
    frame #16: 0x00007ffff506faa1 MyPlugin.so`juce::MessageThread::start(this=0x00005555567c9c38)::'lambda'()::operator()() const at juce_LinuxMessageThread.h:70:23
    frame #17: 0x00007ffff506fa1d MyPlugin.so`void std::__invoke_impl<void, juce::MessageThread::start()::'lambda'()>((null)=__invoke_other @ 0x00007fffe7b75d68, __f=0x00005555567c9c38)::'lambda'()&&) at invoke.h:61:14
    frame #18: 0x00007ffff506f9ad MyPlugin.so`std::__invoke_result<juce::MessageThread::start()::'lambda'()>::type std::__invoke<juce::MessageThread::start(__fn=0x00005555567c9c38)::'lambda'()>(juce::MessageThread::start()::'lambda'()&&) at invoke.h:96:14
    frame #19: 0x00007ffff506f985 MyPlugin.so`void std::thread::_Invoker<std::tuple<juce::MessageThread::start()::'lambda'()> >::_M_invoke<0ul>(this=0x00005555567c9c38, (null)=_Index_tuple<0UL> @ 0x00007fffe7b75da8) at std_thread.h:253:13
    frame #20: 0x00007ffff506f955 MyPlugin.so`std::thread::_Invoker<std::tuple<juce::MessageThread::start()::'lambda'()> >::operator(this=0x00005555567c9c38)() at std_thread.h:260:11
    frame #21: 0x00007ffff506f8b9 MyPlugin.so`std::thread::_State_impl<std::thread::_Invoker<std::tuple<juce::MessageThread::start()::'lambda'()> > >::_M_run(this=0x00005555567c9c30) at std_thread.h:211:13
    frame #22: 0x00007ffff7c952c3 libstdc++.so.6`___lldb_unnamed_symbol7823 + 19
    frame #23: 0x00007ffff791eb43 libc.so.6`start_thread(arg=<unavailable>) at pthread_create.c:442:8
    frame #24: 0x00007ffff79b0a00 libc.so.6`__clone3 at clone3.S:81
3

I’m not an expert on this topic, but it looks like Godot had similar issues — Atoms needed to be checked for existence before calling X_ChangeProperty.

I’m looking into pairing Xvfb/Xdummy with a window manager to see if that works around the issue…

1 Like
4

Be careful that you’re misinterpreting the values returned from X_ChangeProperty - use error handlers after X_* calls to validate error state, and be aware that xlib functions return 1 most of the time - this can be misinterpreted as a ‘fail because not=0’, but its there because you’re expected to use error handlers to check state:

https://tronche.com/gui/x/xlib/event-handling/protocol-errors/default-handlers.html

XLib can be a constant source of frustration if you forget that you are supposed to check things almost every step of the way and not rely on function-returned values…

1 Like
5

Appreciate those insights Jay!

Things seem happier with a window manager (fluxbox), but still not out of the woods…It does seem that JUCE’s XLib implementation would benefit from better error handling, as you described.

6

Thanks for reporting. I can reproduce a very similar issue on a full desktop environment, so having a headless installation seems to be incidental. The key for me was to use --validate-in-process parameter. I’m still trying to figure out what goes wrong exactly.

1 Like
7

I managed to reproduce very similar crashes to this with a plugin that was creating a separate window of its own.

In my case the problem when running pluginval in command line was that the plugin would start calling Xlib functions before proper initialisation by the host. In particular it was missing the call to XInitThreads() that seemed to be the main culprit.

Xlib is initialised lazily to support use cases where X11 is not available. So I think it is the host’s responsibility to do explicit initialisation, when it knows that it’s about to do things that require it, such as opening a plugin that may use it.

A direct way of doing this would be adding the JUCE_GUI_BASICS_INCLUDE_XHEADERS=1 definition and calling juce::XWindowSystem::getInstance(). The extra includes however cause compile errors in pluginval, so a workaround is making a call that will certainly initialise Xlib.

std::make_unique<DocumentWindow> ("", Colours::black, DocumentWindow::allButtons);

Placing this line at the beginning of PluginValidatorApplication::initialise() fixed the crash for me. I hope this will work for you too.

2 Likes
8

Hey Attila, thanks so much for looking into this! You are awesome. I’m super appreciative that you went out of your way to check into pluginval, which isn’t even a part of JUCE core.

So I think it is the host’s responsibility to do explicit initialization, when it knows that it’s about to do things that require it, such as opening a plugin that may use it.

This sounds reasonable to me.

I’ll take a look to see if it resolves the issues here (I’m betting it will) and if so, I’ll submit a PR to pluginval.

Thanks again!

1 Like
9

Another delayed thank you to you @attila! I ended up resolving the compile errors and adding the JUCE_GUI_BASICS_INCLUDE_XHEADERS=1 — a perfect resolution and pluginval seems to finally be all the way happy on linux with Xvfb.

Esoterically, Xlib.h is doing a somewhat evil #define Status int which conflicted with a pluginval enum class.

1 Like