Data race when switching audio device sample rate

Thread sanitizer is reporting a data race when switching the sample rate of a running audio device. Sometimes the application crashes with a sefault.

Environment:
macOS 14.6.1 (23G93)
JUCE 8.0.6

Steps to reproduce:

  • Start an audio device using juce::AudioDeviceManager
  • While processing, switch the sample rate of the device using juce::AudioDeviceSettingsPanel

Report from TSAN:

==================
WARNING: ThreadSanitizer: data race (pid=71223)
  Write of size 8 at 0x00016fa28f18 by main thread:
    #0 juce::HeapBlock<unsigned int, false>::HeapBlock() juce_HeapBlock.h:371 (RAVENNAKIT JUCE Demo:arm64+0x1001b30a4)
    #1 juce::HeapBlock<unsigned int, false>::HeapBlock() juce_HeapBlock.h:112 (RAVENNAKIT JUCE Demo:arm64+0x1001143d8)
    #2 juce::BigInteger::BigInteger(juce::BigInteger const&) juce_BigInteger.cpp:115 (RAVENNAKIT JUCE Demo:arm64+0x100114b0c)
    #3 juce::BigInteger::BigInteger(juce::BigInteger const&) juce_BigInteger.cpp:119 (RAVENNAKIT JUCE Demo:arm64+0x100114f04)
    #4 auto juce::CoreAudioClasses::CoreAudioInternal::getWithDefault<juce::BigInteger const>(std::__1::unique_ptr<juce::CoreAudioClasses::CoreAudioInternal::Stream, std::__1::unique_ptr::default_delete<std::__1::unique_ptr::default_delete>> const&, juce::BigInteger const std::__1::unique_ptr::default_delete::*)::'lambda'(std::__1::unique_ptr::default_delete&)::operator()('lambda'(std::__1::unique_ptr::default_delete&)) const juce_CoreAudio_mac.cpp:1038 (RAVENNAKIT JUCE Demo:arm64+0x10147b1cc)
    #5 std::__1 juce::CoreAudioClasses::CoreAudioInternal::getWithDefault<auto juce::CoreAudioClasses::CoreAudioInternal::getWithDefault<juce::BigInteger const>(std::__1::unique_ptr<juce::CoreAudioClasses::CoreAudioInternal::Stream, std::__1::unique_ptr::default_delete<std::__1::unique_ptr::default_delete>> const&, juce::BigInteger const std::__1::unique_ptr::default_delete::*)::'lambda'(std::__1::unique_ptr::default_delete&)>(juce::BigInteger const, juce::BigInteger const std::__1::unique_ptr::default_delete::*&&) juce_CoreAudio_mac.cpp:1032 (RAVENNAKIT JUCE Demo:arm64+0x10147b118)
    #6 auto juce::CoreAudioClasses::CoreAudioInternal::getWithDefault<juce::BigInteger const>(std::__1::unique_ptr<juce::CoreAudioClasses::CoreAudioInternal::Stream, std::__1::unique_ptr::default_delete<std::__1::unique_ptr::default_delete>> const&, juce::BigInteger const std::__1::unique_ptr::default_delete::*) juce_CoreAudio_mac.cpp:1038 (RAVENNAKIT JUCE Demo:arm64+0x10147b080)
    #7 juce::CoreAudioClasses::CoreAudioInternal::getActiveChannels(std::__1::unique_ptr<juce::CoreAudioClasses::CoreAudioInternal::Stream, std::__1::default_delete<juce::CoreAudioClasses::CoreAudioInternal::Stream>> const&) juce_CoreAudio_mac.cpp:1046 (RAVENNAKIT JUCE Demo:arm64+0x10146f400)
    #8 juce::CoreAudioClasses::CoreAudioIODevice::getActiveOutputChannels() const juce_CoreAudio_mac.cpp:1321 (RAVENNAKIT JUCE Demo:arm64+0x101466c84)
    #9 juce::CoreAudioClasses::AudioIODeviceCombiner::DeviceWrapper::getActiveChannels() const juce_CoreAudio_mac.cpp:2001 (RAVENNAKIT JUCE Demo:arm64+0x101483d60)
    #10 juce::CoreAudioClasses::AudioIODeviceCombiner::getActiveOutputChannels() const juce_CoreAudio_mac.cpp:1491 (RAVENNAKIT JUCE Demo:arm64+0x101481388)
    #11 juce::AudioDeviceManager::updateCurrentSetup() juce_AudioDeviceManager.cpp:189 (RAVENNAKIT JUCE Demo:arm64+0x10143bda0)
    #12 juce::AudioDeviceManager::setAudioDeviceSetup(juce::AudioDeviceManager::AudioDeviceSetup const&, bool) juce_AudioDeviceManager.cpp:824 (RAVENNAKIT JUCE Demo:arm64+0x10143edd4)
    #13 juce::AudioDeviceSettingsPanel::updateConfig(bool, bool, bool, bool) juce_AudioDeviceSelectorComponent.cpp:452 (RAVENNAKIT JUCE Demo:arm64+0x10108bae0)
    #14 juce::AudioDeviceSettingsPanel::updateSampleRateComboBox(juce::AudioIODevice*)::'lambda'()::operator()() const juce_AudioDeviceSelectorComponent.cpp:782 (RAVENNAKIT JUCE Demo:arm64+0x10109e378)
    #15 decltype(std::declval<juce::AudioDeviceSettingsPanel::updateSampleRateComboBox(juce::AudioIODevice*)::'lambda'()&>()()) std::__1::__invoke[abi:ne180100]<juce::AudioDeviceSettingsPanel::updateSampleRateComboBox(juce::AudioIODevice*)::'lambda'()&>(juce::AudioDeviceSettingsPanel::updateSampleRateComboBox(juce::AudioIODevice*)::'lambda'()&) invoke.h:344 (RAVENNAKIT JUCE Demo:arm64+0x10109e2fc)
    #16 void std::__1::__invoke_void_return_wrapper<void, true>::__call[abi:ne180100]<juce::AudioDeviceSettingsPanel::updateSampleRateComboBox(juce::AudioIODevice*)::'lambda'()&>(juce::AudioDeviceSettingsPanel::updateSampleRateComboBox(juce::AudioIODevice*)::'lambda'()&) invoke.h:419 (RAVENNAKIT JUCE Demo:arm64+0x10109e260)
    #17 std::__1::__function::__alloc_func<juce::AudioDeviceSettingsPanel::updateSampleRateComboBox(juce::AudioIODevice*)::'lambda'(), std::__1::allocator<juce::AudioDeviceSettingsPanel::updateSampleRateComboBox(juce::AudioIODevice*)::'lambda'()>, void ()>::operator()[abi:ne180100]() function.h:169 (RAVENNAKIT JUCE Demo:arm64+0x10109e20c)
    #18 std::__1::__function::__func<juce::AudioDeviceSettingsPanel::updateSampleRateComboBox(juce::AudioIODevice*)::'lambda'(), std::__1::allocator<juce::AudioDeviceSettingsPanel::updateSampleRateComboBox(juce::AudioIODevice*)::'lambda'()>, void ()>::operator()() function.h:311 (RAVENNAKIT JUCE Demo:arm64+0x10109c15c)
    #19 std::__1::__function::__value_func<void ()>::operator()[abi:ne180100]() const function.h:428 (RAVENNAKIT JUCE Demo:arm64+0x100068e74)
    #20 std::__1::function<void ()>::operator()() const function.h:981 (RAVENNAKIT JUCE Demo:arm64+0x1000689ec)
    #21 void juce::NullCheckedInvocation::invoke<std::__1::function<void ()>&>(std::__1::function<void ()>&) juce_Functional.h:71 (RAVENNAKIT JUCE Demo:arm64+0x100387bd4)
    #22 juce::ComboBox::handleAsyncUpdate() juce_ComboBox.cpp:637 (RAVENNAKIT JUCE Demo:arm64+0x100486c98)
    #23 non-virtual thunk to juce::ComboBox::handleAsyncUpdate() juce_ComboBox.cpp (RAVENNAKIT JUCE Demo:arm64+0x100486df8)
    #24 juce::AsyncUpdater::AsyncUpdaterMessage::messageCallback() juce_AsyncUpdater.cpp:46 (RAVENNAKIT JUCE Demo:arm64+0x100fb4f28)
    #25 juce::MessageQueue::deliverNextMessage() juce_MessageQueue_mac.h:93 (RAVENNAKIT JUCE Demo:arm64+0x100fe0cd0)
    #26 juce::MessageQueue::runLoopCallback() juce_MessageQueue_mac.h:104 (RAVENNAKIT JUCE Demo:arm64+0x100fe0c00)
    #27 juce::MessageQueue::runLoopSourceCallback(void*) juce_MessageQueue_mac.h:112 (RAVENNAKIT JUCE Demo:arm64+0x100fe06cc)
    #28 __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__ <null>:130030144 (CoreFoundation:arm64e+0x7e4d4)
    #29 juce::JUCEApplicationBase::main() juce_ApplicationBase.cpp:277 (RAVENNAKIT JUCE Demo:arm64+0x100f9ffd0)
    #30 juce::JUCEApplicationBase::main(int, char const**) juce_ApplicationBase.cpp:255 (RAVENNAKIT JUCE Demo:arm64+0x100f9fd7c)
    #31 main MainApplication.cpp:173 (RAVENNAKIT JUCE Demo:arm64+0x10000dfb4)

  Previous read of size 8 at 0x00016fa28f18 by thread T59 (mutexes: write M0):
    #0 juce::CoreAudioClasses::AudioIODeviceCombiner::shutdown(juce::String const&) juce_CoreAudio_mac.cpp:1752 (RAVENNAKIT JUCE Demo:arm64+0x10148e498)
    #1 juce::CoreAudioClasses::AudioIODeviceCombiner::stop() juce_CoreAudio_mac.cpp:1695 (RAVENNAKIT JUCE Demo:arm64+0x101481124)
    #2 juce::CoreAudioClasses::AudioIODeviceCombiner::close() juce_CoreAudio_mac.cpp:1583 (RAVENNAKIT JUCE Demo:arm64+0x101480cec)
    #3 juce::CoreAudioClasses::AudioIODeviceCombiner::restartAsync() juce_CoreAudio_mac.cpp:1647 (RAVENNAKIT JUCE Demo:arm64+0x101481664)
    #4 non-virtual thunk to juce::CoreAudioClasses::AudioIODeviceCombiner::restartAsync() juce_CoreAudio_mac.cpp (RAVENNAKIT JUCE Demo:arm64+0x1014817f8)
    #5 juce::CoreAudioClasses::CoreAudioIODevice::restart() juce_CoreAudio_mac.cpp:1376 (RAVENNAKIT JUCE Demo:arm64+0x10147b8bc)
    #6 juce::CoreAudioClasses::CoreAudioInternal::deviceRequestedRestart() juce_CoreAudio_mac.cpp:857 (RAVENNAKIT JUCE Demo:arm64+0x10147b560)
    #7 juce::CoreAudioClasses::CoreAudioInternal::deviceListenerProc(unsigned int, unsigned int, AudioObjectPropertyAddress const*, void*) juce_CoreAudio_mac.cpp:1194 (RAVENNAKIT JUCE Demo:arm64+0x10146881c)
    #8 HALObject::PropertiesChanged(unsigned int, AudioObjectPropertyAddress const*) <null>:130030224 (CoreAudio:arm64e+0x269b8c)
    #9 _dispatch_client_callout <null>:126121008 (libdispatch.dylib:arm64e+0x43e4)

  Location is stack of main thread.

  Mutex M0 (0x000109c0c118) created at:
    #0 pthread_mutex_init <null>:134225072 (libclang_rt.tsan_osx_dynamic.dylib:arm64e+0x3181c)
    #1 juce::CriticalSection::CriticalSection() juce_SharedCode_posix.h:46 (RAVENNAKIT JUCE Demo:arm64+0x10012cbfc)
    #2 juce::CriticalSection::CriticalSection() juce_SharedCode_posix.h:39 (RAVENNAKIT JUCE Demo:arm64+0x1000fb048)
    #3 juce::CoreAudioClasses::AudioIODeviceCombiner::AudioIODeviceCombiner(juce::String const&, juce::CoreAudioClasses::CoreAudioIODeviceType*, std::__1::unique_ptr<juce::CoreAudioClasses::CoreAudioIODevice, std::__1::default_delete<juce::CoreAudioClasses::CoreAudioIODevice>>&&, std::__1::unique_ptr<juce::CoreAudioClasses::CoreAudioIODevice, std::__1::default_delete<juce::CoreAudioClasses::CoreAudioIODevice>>&&) juce_CoreAudio_mac.cpp:1461 (RAVENNAKIT JUCE Demo:arm64+0x10147fd48)
    #4 juce::CoreAudioClasses::AudioIODeviceCombiner::AudioIODeviceCombiner(juce::String const&, juce::CoreAudioClasses::CoreAudioIODeviceType*, std::__1::unique_ptr<juce::CoreAudioClasses::CoreAudioIODevice, std::__1::default_delete<juce::CoreAudioClasses::CoreAudioIODevice>>&&, std::__1::unique_ptr<juce::CoreAudioClasses::CoreAudioIODevice, std::__1::default_delete<juce::CoreAudioClasses::CoreAudioIODevice>>&&) juce_CoreAudio_mac.cpp:1470 (RAVENNAKIT JUCE Demo:arm64+0x10147fab4)
    #5 std::__1::__unique_if<juce::CoreAudioClasses::AudioIODeviceCombiner>::__unique_single std::__1::make_unique[abi:ne180100]<juce::CoreAudioClasses::AudioIODeviceCombiner, juce::String&, juce::CoreAudioClasses::CoreAudioIODeviceType*, std::__1::unique_ptr<juce::CoreAudioClasses::CoreAudioIODevice, std::__1::default_delete<juce::CoreAudioClasses::CoreAudioIODevice>>, std::__1::unique_ptr<juce::CoreAudioClasses::CoreAudioIODevice, std::__1::default_delete<juce::CoreAudioClasses::CoreAudioIODevice>>>(juce::String&, juce::CoreAudioClasses::CoreAudioIODeviceType*&&, std::__1::unique_ptr<juce::CoreAudioClasses::CoreAudioIODevice, std::__1::default_delete<juce::CoreAudioClasses::CoreAudioIODevice>>&&, std::__1::unique_ptr<juce::CoreAudioClasses::CoreAudioIODevice, std::__1::default_delete<juce::CoreAudioClasses::CoreAudioIODevice>>&&) unique_ptr.h:597 (RAVENNAKIT JUCE Demo:arm64+0x101465658)
    #6 juce::CoreAudioClasses::CoreAudioIODeviceType::createDevice(juce::String const&, juce::String const&) juce_CoreAudio_mac.cpp:2262 (RAVENNAKIT JUCE Demo:arm64+0x10145ee8c)
    #7 juce::AudioDeviceManager::setAudioDeviceSetup(juce::AudioDeviceManager::AudioDeviceSetup const&, bool) juce_AudioDeviceManager.cpp:774 (RAVENNAKIT JUCE Demo:arm64+0x10143e9e8)
    #8 juce::AudioDeviceManager::initialiseDefault(juce::String const&, juce::AudioDeviceManager::AudioDeviceSetup const*) juce_AudioDeviceManager.cpp:400 (RAVENNAKIT JUCE Demo:arm64+0x10143cd4c)
    #9 juce::AudioDeviceManager::initialise(int, int, juce::XmlElement const*, bool, juce::String const&, juce::AudioDeviceManager::AudioDeviceSetup const*) juce_AudioDeviceManager.cpp:322 (RAVENNAKIT JUCE Demo:arm64+0x10143de50)
    #10 MainApplication::initialise(juce::String const&) MainApplication.cpp:98 (RAVENNAKIT JUCE Demo:arm64+0x10000b60c)
    #11 juce::JUCEApplicationBase::initialiseApp() juce_ApplicationBase.cpp:312 (RAVENNAKIT JUCE Demo:arm64+0x100fa06a4)
    #12 juce::JUCEApplication::initialiseApp() juce_Application.cpp:97 (RAVENNAKIT JUCE Demo:arm64+0x100382ad0)
    #13 juce::JUCEApplicationBase::main() juce_ApplicationBase.cpp:271 (RAVENNAKIT JUCE Demo:arm64+0x100f9ff80)
    #14 juce::JUCEApplicationBase::main(int, char const**) juce_ApplicationBase.cpp:255 (RAVENNAKIT JUCE Demo:arm64+0x100f9fd7c)
    #15 main MainApplication.cpp:173 (RAVENNAKIT JUCE Demo:arm64+0x10000dfb4)

  Thread T59 (tid=5395881, running) is a GCD worker thread

SUMMARY: ThreadSanitizer: data race juce_HeapBlock.h:371 in juce::HeapBlock<unsigned int, false>::HeapBlock()
1 Like

Here is another trace:

==================
WARNING: ThreadSanitizer: data race (pid=36450)
  Read of size 4 at 0x000109d15430 by thread T11:
    #0 juce::Timer::startTimer(int) juce_Timer.cpp:378 (RAVENNAKIT JUCE Demo:arm64+0x10100fd68)
    #1 juce::CoreAudioClasses::AudioIODeviceCombiner::restartAsync() juce_CoreAudio_mac.cpp:1651 (RAVENNAKIT JUCE Demo:arm64+0x1014cd4d4)
    #2 non-virtual thunk to juce::CoreAudioClasses::AudioIODeviceCombiner::restartAsync() juce_CoreAudio_mac.cpp (RAVENNAKIT JUCE Demo:arm64+0x1014cd630)
    #3 juce::CoreAudioClasses::CoreAudioIODevice::restart() juce_CoreAudio_mac.cpp:1376 (RAVENNAKIT JUCE Demo:arm64+0x1014c78ec)
    #4 juce::CoreAudioClasses::CoreAudioInternal::deviceRequestedRestart() juce_CoreAudio_mac.cpp:857 (RAVENNAKIT JUCE Demo:arm64+0x1014c75ac)
    #5 juce::CoreAudioClasses::CoreAudioInternal::deviceListenerProc(unsigned int, unsigned int, AudioObjectPropertyAddress const*, void*) juce_CoreAudio_mac.cpp:1194 (RAVENNAKIT JUCE Demo:arm64+0x1014b5a38)
    #6 HALObject::PropertiesChanged(unsigned int, AudioObjectPropertyAddress const*) <null> (CoreAudio:arm64e+0x27f9bc)
    #7 _dispatch_client_callout <null> (libdispatch.dylib:arm64e+0x1b858)

  Previous write of size 4 at 0x000109d15430 by thread T4:
    #0 juce::Timer::startTimer(int) juce_Timer.cpp:379 (RAVENNAKIT JUCE Demo:arm64+0x10100fd9c)
    #1 juce::CoreAudioClasses::AudioIODeviceCombiner::restartAsync() juce_CoreAudio_mac.cpp:1651 (RAVENNAKIT JUCE Demo:arm64+0x1014cd4d4)
    #2 non-virtual thunk to juce::CoreAudioClasses::AudioIODeviceCombiner::restartAsync() juce_CoreAudio_mac.cpp (RAVENNAKIT JUCE Demo:arm64+0x1014cd630)
    #3 juce::CoreAudioClasses::CoreAudioIODevice::restart() juce_CoreAudio_mac.cpp:1376 (RAVENNAKIT JUCE Demo:arm64+0x1014c78ec)
    #4 juce::CoreAudioClasses::CoreAudioInternal::deviceRequestedRestart() juce_CoreAudio_mac.cpp:857 (RAVENNAKIT JUCE Demo:arm64+0x1014c75ac)
    #5 juce::CoreAudioClasses::CoreAudioInternal::deviceListenerProc(unsigned int, unsigned int, AudioObjectPropertyAddress const*, void*) juce_CoreAudio_mac.cpp:1194 (RAVENNAKIT JUCE Demo:arm64+0x1014b5a38)
    #6 HALObject::PropertiesChanged(unsigned int, AudioObjectPropertyAddress const*) <null> (CoreAudio:arm64e+0x27f9bc)
    #7 _dispatch_client_callout <null> (libdispatch.dylib:arm64e+0x1b858)

  Location is heap block of size 1056 at 0x000109d15400 allocated by main thread:
    #0 operator new(unsigned long) <null> (libclang_rt.tsan_osx_dynamic.dylib:arm64e+0x8a6b0)
    #1 std::__1::__unique_if<juce::CoreAudioClasses::AudioIODeviceCombiner>::__unique_single std::__1::make_unique[abi:ne190102]<juce::CoreAudioClasses::AudioIODeviceCombiner, juce::String&, juce::CoreAudioClasses::CoreAudioIODeviceType*, std::__1::unique_ptr<juce::CoreAudioClasses::CoreAudioIODevice, std::__1::default_delete<juce::CoreAudioClasses::CoreAudioIODevice>>, std::__1::unique_ptr<juce::CoreAudioClasses::CoreAudioIODevice, std::__1::default_delete<juce::CoreAudioClasses::CoreAudioIODevice>>>(juce::String&, juce::CoreAudioClasses::CoreAudioIODeviceType*&&, std::__1::unique_ptr<juce::CoreAudioClasses::CoreAudioIODevice, std::__1::default_delete<juce::CoreAudioClasses::CoreAudioIODevice>>&&, std::__1::unique_ptr<juce::CoreAudioClasses::CoreAudioIODevice, std::__1::default_delete<juce::CoreAudioClasses::CoreAudioIODevice>>&&) unique_ptr.h:635 (RAVENNAKIT JUCE Demo:arm64+0x1014b28ec)
    #2 juce::CoreAudioClasses::CoreAudioIODeviceType::createDevice(juce::String const&, juce::String const&) juce_CoreAudio_mac.cpp:2262 (RAVENNAKIT JUCE Demo:arm64+0x1014ac264)
    #3 juce::AudioDeviceManager::setAudioDeviceSetup(juce::AudioDeviceManager::AudioDeviceSetup const&, bool) juce_AudioDeviceManager.cpp:774 (RAVENNAKIT JUCE Demo:arm64+0x10148c20c)
    #4 juce::AudioDeviceManager::initialiseDefault(juce::String const&, juce::AudioDeviceManager::AudioDeviceSetup const*) juce_AudioDeviceManager.cpp:400 (RAVENNAKIT JUCE Demo:arm64+0x10148a5cc)
    #5 juce::AudioDeviceManager::initialise(int, int, juce::XmlElement const*, bool, juce::String const&, juce::AudioDeviceManager::AudioDeviceSetup const*) juce_AudioDeviceManager.cpp:322 (RAVENNAKIT JUCE Demo:arm64+0x10148b684)
    #6 MainApplication::initialise(juce::String const&) MainApplication.cpp:81 (RAVENNAKIT JUCE Demo:arm64+0x10000a028)
    #7 juce::JUCEApplicationBase::initialiseApp() juce_ApplicationBase.cpp:312 (RAVENNAKIT JUCE Demo:arm64+0x101008974)
    #8 juce::JUCEApplication::initialiseApp() juce_Application.cpp:97 (RAVENNAKIT JUCE Demo:arm64+0x100453758)
    #9 juce::JUCEApplicationBase::main() juce_ApplicationBase.cpp:271 (RAVENNAKIT JUCE Demo:arm64+0x10100826c)
    #10 juce::JUCEApplicationBase::main(int, char const**) juce_ApplicationBase.cpp:255 (RAVENNAKIT JUCE Demo:arm64+0x101008070)
    #11 main MainApplication.cpp:307 (RAVENNAKIT JUCE Demo:arm64+0x100011300)

  Thread T11 (tid=1237468, running) is a GCD worker thread

  Thread T4 (tid=1237354, running) is a GCD worker thread

SUMMARY: ThreadSanitizer: data race juce_Timer.cpp:378 in juce::Timer::startTimer(int)
==================

Anyone?

I’ve unfortunately not had any luck trying to reproduce this with the DemoRunner.

Looking at the original report, it looks like while changing the sample rate a callback comes in and there is a race. It’s possible this was dealt with in a commit which was added to JUCE 8.0.8 https://github.com/juce-framework/JUCE-dev/commit/72c7fd30708563414f75636691a2748434db5046

The other report is more surprising. It seems CoreAudio is triggering the same callback for the same device on two different threads! Are you using an aggregate device? Anything else you can think of that’s special about what you’re doing?

1 Like

Actually looking more closely I think you must be using an aggregate device I missed this was happening in the AudioIODeviceCombiner. So I think it’s probably that we’re not dealing with the fact that we might get callbacks from both devices at once. I’ll take a look.

1 Like

@ruurdadema if you’re able to easily produce the issue can you please try this patch.

JUCE Timer: Make timer thread safe.patch (3.7 KB)

1 Like

Hi Anthony,

Thank for picking this up.

I have created a minimal reproducible example:

I’m getting a TSAN hit rate of at least 95%.

Unfortunately your patch didn’t solve the problem.

I think it’s necessary to use 2 different devices to hit this problem.

Let me know!

Edit: I’m not using an aggregate device.

2 Likes

FWIW I have been chasing a crash (segfault) that I think is related to this as well. For me it happens very rarely when my standalone app starts up and the audio device is first being configured. Very difficult to reproduce, but I have a couple of stack traces like the one below.

Crashed Thread:        1  Dispatch queue: HALC_ShellObject_Listener Queue

Exception Type:        EXC_BAD_ACCESS (SIGSEGV)
Exception Codes:       KERN_INVALID_ADDRESS at 0x0000000000000028
Exception Codes:       0x0000000000000001, 0x0000000000000028

Termination Reason:    Namespace SIGNAL, Code 11 Segmentation fault: 11
Terminating Process:   exc handler [35147]

VM Region Info: 0x28 is not in any region.  Bytes before following region: 4375199704
      REGION TYPE                    START - END         [ VSIZE] PRT/MAX SHRMOD  REGION DETAIL
      UNUSED SPACE AT START
--->  
      __TEXT                      104c84000-106148000    [ 20.8M] r-x/r-x SM=COW  /Users/USER/*/Anukari.app/Contents/MacOS/Anukari

Thread 0:
0   libsystem_pthread.dylib             0x196c0eb6c start_wqthread + 0

Thread 1 Crashed::  Dispatch queue: HALC_ShellObject_Listener Queue
0   Anukari                             0x105272db0 juce::CoreAudioClasses::AudioIODeviceCombiner::shutdown(juce::String const&) + 268
1   Anukari                             0x105270ec0 juce::CoreAudioClasses::AudioIODeviceCombiner::close() + 40
2   Anukari                             0x105271478 non-virtual thunk to juce::CoreAudioClasses::AudioIODeviceCombiner::restartAsync() + 56
3   Anukari                             0x10526d0c8 juce::CoreAudioClasses::CoreAudioInternal::deviceListenerProc(unsigned int, unsigned int, AudioObjectPropertyAddress const*, void*) + 532
4   CoreAudio                           0x199b8eaa4 HALObject::PropertiesChanged(unsigned int, AudioObjectPropertyAddress const*) + 1920
5   CoreAudio                           0x1999ef9c8 HALSystem::AudioObjectPropertiesChanged(AudioHardwarePlugInInterface**, unsigned int, unsigned int, AudioObjectPropertyAddress const*) + 284
6   CoreAudio                           0x199c1ee8c void applesauce::dispatch::v1::async<HALC_ShellObject::PropertiesChanged(unsigned int, unsigned int, AudioObjectPropertyAddress const*, bool) const::$_0&>(NSObject<OS_dispatch_queue>*, HALC_ShellObject::PropertiesChanged(unsigned int, unsigned int, AudioObjectPropertyAddress const*, bool) const::$_0&)::'lambda'(void*)::__invoke(void*) + 64
7   libdispatch.dylib                   0x196a7585c _dispatch_client_callout + 16
8   libdispatch.dylib                   0x196a64350 _dispatch_lane_serial_drain + 740
9   libdispatch.dylib                   0x196a64e60 _dispatch_lane_invoke + 440
10  libdispatch.dylib                   0x196a66170 _dispatch_workloop_invoke + 1612
11  libdispatch.dylib                   0x196a6f264 _dispatch_root_queue_drain_deferred_wlh + 292
12  libdispatch.dylib                   0x196a6eae8 _dispatch_workloop_worker_thread + 540
13  libsystem_pthread.dylib             0x196c0fe64 _pthread_wqthread + 292
14  libsystem_pthread.dylib             0x196c0eb74 start_wqthread + 8

Looking through the AudioIODeviceCombiner code, I notice that AudioIODeviceCombiner::restartAsync() accesses the “callback” member without acquiring the callbackLock first. Could that be the problem? I’m not quite sure how that would cause this crash, but certainly seems like a race to me.

Also it seems like the access of “previousCallback” in AudioIODeviceCombiner::timerCallback should be guarded by the callbackLock?

Again, I don’t clearly see how this would cause the issues reported in this thread, but I’m also not positive it couldn’t cause these issues, and it would be good to rule out any of these racey unguarded member variable accesses.

I’m actively working on changes in this area at the moment.

2 Likes

I think I finally found a way to reproduce the issue, and I found a solution at least for my app, which might also inform a more general solution.

My app uses a AudioProcessorPlayer and AudioDeviceManager to play sound via an AudioProcessor.

I need to customize things heavily, so I don’t use StandaloneFilterWindow, I rolled my own. Anyways, I discovered that if I initialize things in this order, I get crashes at startup when the system sample rate for the default device was changed while my app was not running:

device_manager.initialize(…);
player.setProcessor(processor);

However if I reverse the order of those two statements it never crashes:

player.setProcessor(processor);
device_manager.initialize(…);

From what I can tell, what’s happening is that AudioDeviceManager::initialize() is calling audioDeviceAboutToStart(), and when the AudioProcessorPlayer’s processor is not set, it never gets initialized. Or, maybe, it only gets initialized in certain cases where audioDeviceAboutToStart() is called again.

Now the thing is, I copied this code from StandaloneFilterWindow. Granted mine is dramatically simpler, but if you look at StandaloneFilterWindow::init(), it does things in the wrong order as well: setupAudioDevices() (which ultimately calls initialize()) is called before startPlaying() (which ultimately calls setProcessor()).

I suspect that fixing this ordering issue in StandaloneFilterWindow will fix this crash for everyone.

Note that I don’t understand why this order crashes and the other order doesn’t. Hopefully someone much more conversant with this code can answer that.

Just to update on this, the changes required to make this more reliable are quite significant so I plan on doing an overhaul of the CoreAudio code for macOS but that will take some time. These issues all seem to stem from the use of different input and output devices. If it is at all possible I would advise avoiding that, and instead if a user needs that functionality it can be achieved by creating an aggregate device in Audio MIDI setup and then selecting the aggregate device for both input and output. This will avoid using the AudioIODeviceCombiner internally. I realise this isn’t a long term solution and we will provide a more robust solution in the future, likely with a bunch of other added benefits too. Thanks again for reporting.

@emezeske just to check the issue you reported regarding the order of initialise() and setProcessor()is that also only when there are different input and output devices?

3 Likes

@anthony-nicholls Thanks for continuing to work on this! I am not surprised that it has unfolded into a bigger project. :sob: That is some pretty subtle code.

Regarding your question, I can say that I have seen the issue when there’s an AudioIODeviceCombiner present, but I have not done much testing in the absence of a combiner, since even the most typical default setup on macOS uses one (selecting the “MacBook Pro Speakers” output device and “MacBook Pro Microphone” input device appears to result in a combiner).

I think trying to explain to users how to make an aggregate device out of the default speakers/microphone devices is probably too tricky and will result in more trouble! Fortunately in my case, changing the initialize() and setProcessor() order appears to have worked around the issue for now so I’m in a good spot.

If I may offer a suggestion as you work on the improved solution: with issues like this I have had a huge amount of success with fuzz testing. In this case, that would be a unit test that randomly creates and destroys audio devices, calling into all their methods randomly from multiple threads, some thousands of times. With careful test design I bet you could write a fuzzer that catches the crashes we’re seeing in the wild by randomly stumbling across the bad orders/timings. I use these kinds of tests in my plugin for similar code where there’s very subtle concurrency considerations, and have caught TONS of races that way.

1 Like