JUCE Timer Crash Reports - Logic 11

[mfritze]

Yes, once you find the underlying issue, it is often obvious how to reproduce it, but till then it might be reproducible on some machines, but not on others.

In the 90s we had an extremely rare Logic Platinum crash resulting in a complete system freeze of Classic Macintosh. We could finally reproduce it – but a single try took more than 6 hours! And the system was completely dead – no MacsBug was alive, so I could only write to the screen pixels to help diagnosing it. I could try about three times a day. That clearly didn’t work.

I eventually figured out how to increase the probability to crash: highest possible sample rate and add as many audio devices as possible. This way I could bring it down to 18 minutes. Much better. Still: die computer was completely dead.

At the end it was 3rd party code overflowing the supervisor stack (Motorola 68k CPU, the stack size was by default only 2kb large) and that froze the system. I added some complicated code to switch during our interrupts temporarily to our own supervisor stack, which was much larger.

[mfritze]

Logic Pro does not go down, because AUHostingServiceXPC_arrow (on Apple Silicon) crashes, which is the host daemon running Audio Units out of process.

In the past Logic Pro would have crashed and users would complain how bad Logic Pro is, despite a certain Audio Unit was crashing. Now you at least see an alert informing you that an Audio Unit was unstable.

BTW: Crashes on closing an Audio Unit are so common, it is actually quite sad. I see hundreds of thousands of crashes when C++ destructors are executed and one thread is still firing and bringing it down. That is not JUCE specific. Termination and threading is often forgotten during the implementation phase.

I also see asserts being left in the code and an ANSI-C abort() is called – resulting in immediate termination of the Audio Unit.

[mfritze]

Commenting myself: the timer crash can probably be reproduced by temporarily increasing the frequency of the timer to something ridiculously small. Or by adding sleep(1) in destructors to delay the shutdown.

[Verbonaut]

Has it been established if plug-ins that have been observed crashing are built with a version of Juce that the maintainers believe all timer issues were resolved in, or are they quite old? Do we have suspected problems in the latest v7 and v8 versions of Juce that need to be looked at further?

[anthony-nicholls]

@ZenithSounds

Ahhhh, ok. I think I understand why this is being misunderstood. I apologize for not being clear enough. No actual ‘crash’ of Logic occurs. Logic does not go down. All that happens is a crash log is generated in the Mac Console app. You have to open the Console application and look at the logs after closing a Logic project to see the generated crash logs. Is that clear now?

Also, try just one instance of the plugin. You do not need but one.

Yeah the crash will be in AUHostingServiceXPC_arrow as pointed out by @mfritze, I was over simplifying things for the sake of the conversation. I have no crashes of this process in my console. And I did also try with one instance.

@mfritze

Commenting myself: the timer crash can probably be reproduced by temporarily increasing the frequency of the timer to something ridiculously small. Or by adding sleep(1) in destructors to delay the shutdown.

Unfortunately I’ve tried with no luck.

@Verbonaut

Has it been established if plug-ins that have been observed crashing are built with a version of Juce that the maintainers believe all timer issues were resolved in, or are they quite old? Do we have suspected problems in the latest v7 and v8 versions of Juce that need to be looked at further?

It has been established in all known cases that an older version of JUCE is being used.

@Verbonaut

Do we have suspected problems in the latest v7 and v8 versions of Juce that need to be looked at further?

No evidence of that yet.

[eyalamir]

Out of interest, I also installed MODO BASS 2 and I can’t get a crash (Latest Sonoma, latest Logic, M1 Max MBP).

But - I also see this version of this plugin (2.0.3) was built recently, in June 2024, maybe it already uses a post-fix version of JUCE?

[ZenithSounds]

Same version of MODO BASS. Still get the crash reports.

Try loading a few other plugins while using MODO BASS. I have Komplete 7 and Superior Drummer also instantiated.

[eyalamir]

OK, this is interesting. When I start a session with two plugins: MODO Bass and Native Instruments Massive X, I am getting a consistent crash in the console on exit.

However - I see no mention of the Timer or JUCE in the crash log.

Attached here:
crashlog.txt (50.2 KB)

[eyalamir]

A few interesting things I’ve learned about ‘my’ crash:
It only happens if I:

1. Load MODO Bass in a new project.
2. Create a new Software Instrument track (this essentially duplicates the MODO bass track).
3. Change the 2nd track to Massive X
4. Quit Logic.

If I change one of these bits in the sequence, like start with Massive, or not add Massive after duplicating, no crash. When I save the project right before quitting, and load it again later - no crash.

This leads me to think there’s something wrong there that isn’t handling refcounting correctly when instances are duplicated and then removed. I can’t see evidence this is specifically on the JUCE Timer side though.

[ZenithSounds]

This is why I believe that Logic plays a role in all of these crashes. I know it has been said before, but I’ve never seen this behavior with any other DAW.

[anthony-nicholls]

I think we might be going of on a tangent here. That crash appears completely unrelated there is no evidence the two are connected. I completely understand it’s importance in sharing it but we should be careful about drawing any conclusions.

But - I also see this version of this plugin (2.0.3) was built recently, in June 2024, maybe it already uses a post-fix version of JUCE?

As I understand it that is not the case, a developer has reached out to me of which I’ve guided them to the commits they would need. However, I think we can all respect that there can be a lot of reasons why the latest version of JUCE might not be in use by any manufacturer.

To summerise

• There is an issue in older versions of JUCE
• There are no reports to date of this happening with modern versions of JUCE
• Some plugins still use this older version of JUCE and therefore are vulnerable to the issue
• Some users appear to be experiencing the issue 100% of the time, others not at all
• Currently there are only reports of this occurring in Logic (but this does not strictly make it the fault of Logic!)

[pflugshaupt]

Could you tell us which JUCE versions are affected? And which commits fixed the issue?
Then we devs could check if our released plugins are a part of the problem or not.

[eyalamir]

In my mind none of this is neither “JUCE’s fault” or “Logic’s fault”. What I’m trying to figure out is the exact behavior a popular host is introducing that a plugin dev needs to be aware of when coding.

Specifically, the interesting parts to me are the ownership patterns when using elements shared with other instances of the plugin - using Singletons and Singleton-like objects such as juce::SharedResourcePointer.

It seems to me like there are situations when the DAW is performing extra steps, such as keeping the dll alive or keeping extra ref counts of instances, and ideally there should be a way to reproduce these edge cases to test my own plugins against.

From what I can observe with the combination of MODO and Massive X, these aren’t specific to JUCE Timer (or at least the crash log isn’t showing clear evidence of that).

[anthony-nicholls]

@eyalamir I understand but just so you can see for yourself this is an example of the crash being referred to

Thread 9 Crashed:: JUCE Timer
0   libsystem_pthread.dylib       	       0x1892044f8 pthread_mutex_lock + 12
1   MODO BASS 2                   	       0x1589e2064 juce::CriticalSection::enter() const + 12
2   MODO BASS 2                   	       0x1589f9d88 juce::MessageQueue::post(juce::MessageManager::MessageBase*) + 28
3   MODO BASS 2                   	       0x1589fac28 juce::Timer::TimerThread::run() + 572
4   MODO BASS 2                   	       0x1589cedb8 juce::Thread::threadEntryPoint() + 320
5   MODO BASS 2                   	       0x1589edeb8 juce::Thread::createNativeThread(juce::Thread::Priority)::$_28::__invoke(void*) + 152
6   libsystem_pthread.dylib       	       0x189209f94 _pthread_start + 136
7   libsystem_pthread.dylib       	       0x189204d34 thread_start + 8

AUHostingServiceXPC_arrow-2024-09-12-093843 2.ips.zip (9.9 KB)

As we can see the “JUCE Timer” thread (which never executes user code it just schedules the user callbacks on the MessageQueue), crashes while trying to lock a mutex, this suggests the queue is probably null at this point so when the mutex is accessed it dereferences it and crashes.

@pflugshaupt the most recent Timer related commit is Timer: Add a leak detector so I would suggest using that or later.

You should be able to find most of the commits by searching Timer: or TimerThread:, it’s worth noting that this wasn’t the only crash surrounding the TimerThread we had and we couldn’t reproduce any of the crashes so most of the this work was an attempt to clean things up as best we could.

Looking back the very first commit I made relating to this was SharedResourcePointer: Get a SharedResourcePointer instance without creating one, so I suspect anything before that should be vulnerable (that’s 7.0.9 or older)

7.0.10 and 7.0.11 might be OK, 7.0.12 had one additional commit, and there are a couple of commits on the JUCE 7 develop branch before 8.0.0 was released but they relate to Windows, preventing an assertion, and adding a leak detector, so unlikely to impact this issue but worth having IMO.

So in other words if you’re on the very last JUCE 7 commit or any commit on JUCE 8 I wouldn’t expect to see this crash.

[Verbonaut]

I’ve had my first instance reported of what appears to be this problem as the stack trace is near identical to the one most recently posted by Anthony. This happened on an Intel machine so is not in the arrow host it’s in-process, Logic 11.0.1, macOS 14.5, the plug-in is built on the latest v7 commit (i.e. 7.0.12 plus the untagged tail pre-v8). I also can’t reproduce it, but if the client is able to provide sufficient information to help reproduce it I’ll dig in deeper. Just posting for the sake of adding context around the in/out of process status, I know there’s nothing firm to go on here.

[anthony-nicholls]

Is it possible you could privately share the crash report with me? Thanks.

[Verbonaut]

I’ve dropped two samples into your inbox.

[ZenithSounds]

The madness continues…

[anthony-nicholls]

That’s not using the latest version of JUCE, in more recent versions we’ve changed the name of the thread to include the JUCE version number, so are you completely sure it includes all the fixes already discussed?

[Verbonaut]

Are there now some Juce 8 changes to the way timers are implemented specifically addressing this issue?

Of course there are some reasons Juce 8 is not so attractive at the moment so a lot of devs probably have not implemented them yet so it’s best not to dismiss this as an old Juce version problem, but I will leave that discussion to other threads of greater relevance.