Coreaudio deadlock when stopping and restarting device

The deadlock happens on my mac mini M1, but not on my macbook 2016 running catalina. At some point the ‘callbackLock’ mutex of juce_mac_CoreAudio.cpp is locked by both the audioCallback() and the start() member functions, and none of them progresses anymore.

It seems that the audioCallback() is called while AudioDeviceStart() is being executed, and AudioDeviceStart() waits for audioCallback to complete, but audioCallback cannot complete because it can’t lock callbackLock

Here is a trace of the last events before the deadlock:

locking audioCallback() ! thread=0x16bedf000 audioDeviceStopPending=0
lock acquired for audioCallback() !
audioCallback() finished!, unlocking

locking audioCallback! thread=0x16bedf000 audioDeviceStopPending=0
lock acquired for audioCallback()!
audioCallback ()finished!, unlocking

locking audioCallback()! thread=0x16bedf000 audioDeviceStopPending=0
lock acquired for audioCallback()!
audioCallback() finished!, unlocking

locking at stop(), leaveInterruptRunning=1
unlocking at stop()
locking at stop(), leaveInterruptRunning=0
 stop(): audioDeviceStopPending=1

locking audioCallback() ! thread=0x16bedf000 audioDeviceStopPending=1
lock acquired for audioCallback()!
returning after AudioDeviceStop() at audioCallback()
 stop(): audioDeviceStopPending=0
unlocking at stop()
locking at stop(), leaveInterruptRunning=0
unlocking at stop()
locking at updateDetailsFromDevice()
unlocking at updateDetailsFromDevice()
locking at stop(), leaveInterruptRunning=0
unlocking at stop()
locking at updateDetailsFromDevice()
unlocking at updateDetailsFromDevice()
locking at updateDetailsFromDevice()
unlocking at updateDetailsFromDevice()
locking at start() thread=0x1057d4580
 started=0

locking audioCallback() ! thread=0x16bedf000 audioDeviceStopPending=0

I have noted that the is always a call to audioCallback with “audioDeviceStopPending=1” just before the deadlock happens.

The relevant part of the stack traces is:

thread #1, name = 'SYNTH.APP', queue = 'com.apple.main-thread'
  * frame #0: 0x00000001a95445c0 libsystem_kernel.dylib`__psynch_mutexwait + 8
    frame #1: 0x00000001a957a364 libsystem_pthread.dylib`_pthread_mutex_firstfit_lock_wait + 84
    frame #2: 0x00000001a9577c98 libsystem_pthread.dylib`_pthread_mutex_firstfit_lock_slow + 240
    frame #3: 0x00000001a957d894 libsystem_pthread.dylib`_pthread_cond_wait + 1368
    frame #4: 0x00000001ab50e41c CoreAudio`HALB_Guard::WaitFor(unsigned long long) + 180
    frame #5: 0x00000001ab2dd308 CoreAudio`HALB_IOThread::_WaitForState(unsigned int) + 184
    frame #6: 0x00000001ab2dd68c CoreAudio`HALB_IOThread::StartAndWaitForState(unsigned int) + 292
    frame #7: 0x00000001ab115d24 CoreAudio`HALC_ProxyIOContext::_StartIO() + 288
    frame #8: 0x00000001ab0dfbcc CoreAudio`HAL_HardwarePlugIn_DeviceStart(AudioHardwarePlugInInterface**, unsigned int, int (*)(unsigned int, AudioTimeStamp const*, AudioBufferList const*, AudioTimeStamp const*, AudioBufferList*, AudioTimeStamp const*, void*)) + 808
    frame #9: 0x00000001ab5071f4 CoreAudio`HALDevice::StartIOProc(int (*)(unsigned int, AudioTimeStamp const*, AudioBufferList const*, AudioTimeStamp const*, AudioBufferList*, AudioTimeStamp const*, void*)) + 104
    frame #10: 0x00000001aaf536b4 CoreAudio`AudioDeviceStart + 524
    frame #11: 0x0000000100907480 juce::CoreAudioClasses::CoreAudioInternal::start(this=0x0000000107f0f550, callbackToNotify=<unavailable>) at juce_mac_CoreAudio.cpp:705:29 [opt]
    frame #12: 0x0000000100904ad0 juce::CoreAudioClasses::CoreAudioIODevice::start(this=0x0000600003e74400, callback=0x0000600000c499b0) at juce_mac_CoreAudio.cpp:1107:23 [opt]
    frame #13: 0x00000001008fedf0 juce::AudioDeviceManager::setAudioDeviceSetup(this=0x0000000102817c50, newSetup=0x000000016fdfe588, treatAsChosenDevice=true) at juce_AudioDeviceManager.cpp:722:29 [opt]
frame #14: 0x00000001008fdcc4 juce::AudioDeviceManager::initialiseFromXML(this=0x0000000102817c50, xml=0x0000600000eab180, selectDefaultDeviceOnFailure=false, preferredDefaultDeviceName=0x0000000102817d60, preferredSetupOptions=0x0000000000000000) at juce_AudioDeviceManager.cpp:428:13 [opt]


thread #14, name = 'com.apple.audio.IOThread.client'
    frame #0: 0x00000001a95445c0 libsystem_kernel.dylib`__psynch_mutexwait + 8
    frame #1: 0x00000001a957a364 libsystem_pthread.dylib`_pthread_mutex_firstfit_lock_wait + 84
    frame #2: 0x00000001a9577c98 libsystem_pthread.dylib`_pthread_mutex_firstfit_lock_slow + 240
    frame #3: 0x000000010082d1f0 juce::CriticalSection::enter(this=<unavailable>) const at juce_posix_SharedCode.h:39:55 [opt]
    frame #4: 0x0000000100907610 juce::CoreAudioClasses::CoreAudioInternal::audioCallback(this=0x0000000107f0f550, inInputData=0x0000600000071be0, outOutputData=0x00006000005c4a60) at juce_mac_CoreAudio.cpp:767:20 [opt]
    frame #5: 0x0000000100907558 juce::CoreAudioClasses::CoreAudioInternal::audioIOProc((null)=<unavailable>, (null)=<unavailable>, inInputData=<unavailable>, (null)=<unavailable>, outOutputData=<unavailable>, (null)=<unavailable>, device=<unavailable>) at juce_mac_CoreAudio.cpp:906:51 [opt]
    frame #6: 0x00000001ab113974 CoreAudio`HALC_ProxyIOContext::IOWorkLoop() + 6288
    frame #7: 0x00000001ab111ae0 CoreAudio`invocation function for block in HALC_ProxyIOContext::HALC_ProxyIOContext(unsigned int, unsigned int) + 100
    frame #8: 0x00000001ab2dd420 CoreAudio`HALB_IOThread::Entry(void*) + 88
    frame #9: 0x00000001a957d240 libsystem_pthread.dylib`_pthread_start + 148

Adding a ScopedUnlock unsl(callbackLock) in start(), just before the if (OK (AudioDeviceStart (deviceID, audioProcID))) call solves the issue, but I guess that is not a “good” fix…

1 Like