CoreAudio (possible) deadlock when moving to Mavericks


#1


I'm getting what looks like a deadlock when moving to Mavericks. When my app starts up it goes through the normal stuff of initialising the GUI and initialising audio via the AudioManager class. However the app often locks up after a few seconds. I've tried running this using the same binary compiled with Xcode 5 on a OSX 10.8 machine, and the problem doesn't occur. The problem is also not visible in Windows. I'll post the details as a new topic so that it shows up as new.


After some investigation I've discovered the following. When the application is starting up, updateDetailsFromDevice() in juce_mac_CoreAudio is being triggered about 5 times (I don't know why, but that's another issue). The routine runs in the message thread because it is not directly called, but is scheduled by a timer from deviceDetailsChanged() which is called in the audio thread context.


The updateDetailsFromDevice()  locks the audio callback with callbackLock and eventually calls the CoreAudio routine AudioObjectGetPropertyData().  Moving a bit further up the stack, eventually HALB_Mutex::Lock() is called and from this point the app hangs.


Meanwhile the audio thread (com.app.audioIOThread.client) is running and calls AudioIOProc which will eventually schedules the callbacks. AudioIOProc is called form CoreAudio by the HALC_ProxyIOContext::IOWorkLoop.  Looking at the dissasembly you can see that this is calling HALB_Mutex::Lock() before it calls AudioIOProc. AudioIOProc first tries to lock callbackLock and hangs.


As described here there are two threads trying to lock the same pair of mutexes, but in reverse order. This is textbook deadlock (I'm assuming that HALB_Mutex::Lock refers to the same mutex in both threads, but I can't find any documentation).


Does anyone have any thoughts in this? I hope I'm wrong because the consequences are nasty if I'm not.


Here is the stack for the two relevant threads at the point of hang

Thread 1 Juce Message Thread, Queue : com.apple.main-thread
#0    0x90688802 in __psynch_mutexwait ()
#1    0x9922b945 in _pthread_mutex_lock ()
#2    0x9922b7ac in pthread_mutex_lock ()
#3    0x9074dc21 in HALB_Mutex::Lock() ()
#4    0x9076a2ed in HALC_ProxyIOContext::GetPropertyDataSize(AudioObjectPropertyAddress const&, unsigned long, void const*, unsigned long&) const ()
#5    0x9077be1a in HALC_ShellObject::GetPropertyDataSize(unsigned long, AudioObjectPropertyAddress const&, unsigned long, void const*, unsigned long&) const ()
#6    0x9077f5a2 in HAL_HardwarePlugIn_ObjectGetPropertyDataSize(AudioHardwarePlugInInterface**, unsigned long, AudioObjectPropertyAddress const*, unsigned long, void const*, unsigned long*) ()
#7    0x9078d045 in HALPlugIn::ObjectGetPropertyDataSize(HALObject const&, AudioObjectPropertyAddress const&, unsigned long, void const*) const ()
#8    0x9078b1fe in HALObject::GetPropertyDataSize(AudioObjectPropertyAddress const&, unsigned long, void const*) const ()
#9    0x90759178 in AudioObjectGetPropertyDataSize ()
#10    0x00056102 in juce::CoreAudioClasses::CoreAudioInternal::fillInChannelInfo(bool)
#11    0x00055a8c in juce::CoreAudioClasses::CoreAudioInternal::updateDetailsFromDevice() 
#12    0x00057d9d in juce::CoreAudioClasses::CoreAudioInternal::timerCallback() 
#13    0x0020e75a in juce::Timer::TimerThread::callTimers() 
#14    0x0020f5e6 in juce::Timer::TimerThread::CallTimersMessage::messageCallback() 
#15    0x0020d5ab in juce::MessageQueue::deliverNextMessage() 
#16    0x0020d4ba in juce::MessageQueue::runLoopCallback() 
#17    0x0020d477 in juce::MessageQueue::runLoopSourceCallback(void*) 
#18    0x9534be3f in __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__ ()
#19    0x9533cccb in __CFRunLoopDoSources0 ()
#20    0x9533c3ce in __CFRunLoopRun ()
#21    0x9533bd5a in CFRunLoopRunSpecific ()
#22    0x9533bbbb in CFRunLoopRunInMode ()
#23    0x91d7ee2d in RunCurrentEventLoopInMode ()
#24    0x91d7ebb2 in ReceiveNextEventCommon ()
#25    0x91d7e98d in _BlockUntilNextEventMatchingListInModeWithFilter ()
#26    0x983e75a9 in _DPSNextEvent ()
#27    0x983e6ad0 in -[NSApplication nextEventMatchingMask:untilDate:inMode:dequeue:] ()
#28    0x983d935c in -[NSApplication run] ()
#29    0x001ff171 in juce::MessageManager::runDispatchLoop() 
#30    0x001ff02a in juce::JUCEApplicationBase::main() 
#31    0x001fedf9 in juce::JUCEApplicationBase::main(int, char const**) 
#32    0x00008610 in main 


Thread 17 com.apple.audio.IOThread.client, Queue : (null)
#0    0x90688802 in __psynch_mutexwait ()
#1    0x9922b945 in _pthread_mutex_lock ()
#2    0x9922b7ac in pthread_mutex_lock ()
#3    0x00187a95 in juce::CriticalSection::enter() const 
#4    0x00006215 in juce::GenericScopedLock<juce::CriticalSection>::GenericScopedLock(juce::CriticalSection const&) 
#5    0x00004974 in juce::GenericScopedLock<juce::CriticalSection>::GenericScopedLock(juce::CriticalSection const&) 
#6    0x00054343 in juce::CoreAudioClasses::CoreAudioInternal::audioCallback(AudioBufferList const*, AudioBufferList*) 
#7    0x000542f2 in juce::CoreAudioClasses::CoreAudioInternal::audioIOProc(unsigned long, AudioTimeStamp const*, AudioBufferList const*, AudioTimeStamp const*, AudioBufferList*, AudioTimeStamp const*, void*)     
#8    0x9075e7b0 in HALC_ProxyIOContext::IOWorkLoop() ()
#9    0x9075d5ff in HALC_ProxyIOContext::IOThreadEntry(void*) ()
#10    0x90767f52 in ___ZN19HALC_ProxyIOContextC2Emj_block_invoke ()
#11    0x9075d4fb in HALB_IOThread::Entry(void*) ()
#12    0x992285fb in _pthread_body ()
#13    0x99228485 in _pthread_start ()

#2

Are you on the latest version? I fixed a problem a few days ago that sounds exactly like this..


#3

Thanks Jules - I can see what's change in updateDetailsFromDevice and that seems to do the trick.  Should have known to check the newest version first. What was the emoticon for shuffling out of the room backwards with an embarrassed expression?


#4

Yes, always worth checking. I've seen a lot of people waste a lot of time over the years posting about bugs that had already been fixed.


#5

@jules,

 

Hi from the LMMS team... 

We have a very similar as described above with the Fluidsynth library and our software on Mac.  Is there a specific place you can recommend looking for resolving this issue?


#6

Hi Tres

This was so long ago that I can't even remember what the original problem entailed, but it has obviously been fixed since 2013! If you're running some old version of juce and are looking for a way to patch it, then sorry, I've no idea what was changed, but you could look through the git log from back around this date, I guess.


#7

Found it, thanks! https://github.com/julianstorer/JUCE/commit/26828713ba418f0271e314c214888c1c0f1c4ea1#diff-80c212bebcf9de54a837d1e31270ca70R371