JUCE 9.0.0 macOS: app deadlocks at launch inside getInputLatencyInSamples (HALC_ProxyObject::HasProperty never returns) on a machine with many audio devices

MacOSX and iOS
1

Filed as [Bug]: JUCE 9.0.0 macOS: app deadlocks at launch inside CoreAudioIODevice::getInputLatencyInSamples (HALC_ProxyObject::HasProperty never returns) on a machine with many audio devices · Issue #1686 · juce-framework/JUCE · GitHub with full sampled call graphs; posting here for visibility since the new CoreAudio implementation is JUCE 9’s headline macOS change.

Short version: since JUCE 9.0.0’s rewritten macOS CoreAudio layer, my JUCE standalone app deadlocks at launch, before its first window appears. The identical codebase on 8.0.13 launches fine on the same machine, same day, same devices. Reproduced twice, sampled both times.

The main thread blocks forever in:

juce::CoreAudioClasses::CoreAudioIODevice::getInputLatencyInSamples()
      juce::AudioObject::getProperty<unsigned int>(...)
        AudioObjectHasProperty (CoreAudio)
          HALC_ShellObject::HasProperty
            HALC_ProxyObject::HasProperty
              mach_msg -> mach_msg2_trap     (100% of samples, both runs)

The synchronous mach message to coreaudiod is never answered. No crash, no window, 0.1 percent CPU, forever.

**Environment:** macOS Ventura 13.7.8, Intel Mac Pro (2019). The app opens a split configuration (loopback driver in, Thunderbolt interface out) and the machine carries about 18 CoreAudio devices (Apogee Symphony I/O Thunderbolt, several virtual loopbacks, six Pro Tools Audio Bridges, NDI, BlackHole, aggregates). My suspicion is the new always-on private aggregate path meeting a large device list: a two-device dev machine likely never sees this, but a crowded pro-audio machine hits it at launch.

Also worth knowing: force-killing the client while it is blocked in that proxy call wedged coreaudiod itself; afterwards even a JUCE 8.0.13 client hung inside AudioDeviceStart until killall coreaudiod.

Ruled out: my code (zero-change compile, full DSP test suite passes against 9.0.0, pluginval strictness 10 passes on the 9.0.0-built VST3 and AU), a one-off (second repro after a clean gap), a broken machine (8.0.13 runs fine between the attempts).

Full details, both complete sample call graphs, and repro steps are in the GitHub issue. Happy to run diagnostic builds.
2

Thanks for reporting this issue. My initial suspicion is a buggy driver, obviously most devices are plug and pay but thunderbolt devices are not. Have you already tried updating to the latest driver?

One reason to suspect this is that the app should not be able cause an issue with coreaudiod itself, I would only expect an issue with coreaudiod or a driver to be able to do that.

Three other things that might be helpful.

  1. Try running up your app with different devices unplugged, probably starting with the Thunderbolt device as that jumps out to me as the most suspicious.

  2. Try setting JUCE_COREAUDIO_LOGGING_ENABLED=1 and sending over the logs. I’m not entirely sure this will reveal a great deal of information given you have already identified that this seems to be stuck in AudioObjectHasProperty.

  3. In juce_audio_devices/natice/juce_CoreAudio_mac.cpp there is a function hasProperty which calls AudioObjectHasProperty. What happens if you make this function always return true so that it never calls AudioObjectHasProperty?

1 Like
3

Thanks Anthony. Ran all three. Short version: removing the Thunderbolt interface makes it launch, and forcing hasProperty() to return true makes it launch with the interface still connected. It is the synchronous property round-trip to that device that hangs.

Machine: macOS Ventura 13.7.8, Intel Mac Pro, 19 CoreAudio devices (Apogee Symphony I/O Mk II over Thunderbolt, three of my own loopback drivers, six Pro Tools Audio Bridge devices, NDI, BlackHole, a couple of aggregates). Selected input is a loopback, selected output is the Symphony.

On the driver: I am on Apogee’s current DEXT-based Thunderbolt driver (com.apogeedigital.TBTDriver 1.0.20, via Apogee Control 2). One thing I noticed while checking: the older kext driver (com.apogeedigital.kextSymphonyIO2T 0.8.7, from before the DEXT migration) is still loaded alongside it, so coreaudiod currently has both an old kext and a new dext registered for this device. 8.0.13 launches fine in exactly this state, 9.0.0 does not. Happy to strip the stale kext and retest if you think the dual-driver state is the smoking gun.

1. Unplug test. With the Symphony I/O powered off and gone from CoreAudio, the app launches every time and reaches its window. Power it back on and it deadlocks at launch every time, before the window appears. The Thunderbolt device is the trigger, exactly as you suspected.

2. hasProperty() always true. Patched AudioObject::hasProperty to return true and never call AudioObjectHasProperty (otherwise stock 9.0.0, logging off). With the Symphony connected it launches, main thread sitting in the normal run loop:

juce::JUCEApplicationBase::main()
  -[NSApplication run]
    CFRunLoopRunSpecific
      __CFRunLoopRun          (idle, app is up)

Skipping that one AudioObjectHasProperty call is enough to get past the deadlock here.

3. JUCE_COREAUDIO_LOGGING_ENABLED=1. Worth knowing before you read the log: turning logging on moves the hang. The AudioObject constructor calls AudioObjectShow(objectId) under #if JUCE_COREAUDIO_LOGGING_ENABLED (juce_CoreAudio_mac.cpp around line 498), which forces the first HAL checkout early, so the logging build wedges in Apple’s own CheckOutInstance sweep instead of getInputLatencyInSamples. Different call site, identical mechanism:

juce::AudioDeviceManager::createAudioDeviceTypes(...)
  juce::CoreAudioClasses::CoreAudioIODeviceType::CoreAudioIODeviceType()
    AudioObjectShow                                        (logging-only)
      HALSystem::CheckOutInstance()
        HALSystem::UpdateInputProcessingNotificationHandler()
          System_Input_Processing_Notification_Handler::set_audio_processing_type_on_all_devices(...)
            HALObject::SetPropertyData / IsPropertySettable / HasProperty   (iterating every device)
              HALC_ProxyObject::SetPropertyData / IsPropertySettable / HasProperty
                mach_msg2_trap        (never returns)

The log enumerates the full 19-device list cleanly then bogs down here. It logs millions of proxy queries and never reaches the run loop. Same wall as the production hang: a HALC_ProxyObject synchronous mach_msg to coreaudiod that never comes back. So it is not one specific selector, it is any synchronous property round-trip to that device once the HAL is checked out with the full device list present.

One more that fits: JUCE 9 builds a private aggregate from the selected input and output (I see _JucePrivateAggregateDevice_<pid> appear). The Symphony is the output member, so queries proxy to the Thunderbolt device, which is the one that stops answering. Remove it and the hang goes away.

Happy to run anything else: a specific patch, a narrower guard, or a clean stock A/B with the device in and out.

4

Following up a week on, in case this one slipped.

Where it stands, briefly: 8.0.13 launches fine on this machine in exactly the state described. 9.0.0 deadlocks at launch every time the Thunderbolt interface is connected, and patching AudioObject::hasProperty to return true without calling AudioObjectHasProperty gets it past the deadlock. So a single synchronous HAL property query during launch is the whole problem, and the Symphony’s HAL proxy is what stops answering it. Reproduces 100% of the time, and I still have the machine in that state.

One question would unblock me regardless of when a fix lands, and it may be a one-liner for you: is the private aggregate that 9.0.0 builds from the selected input and output (_JucePrivateAggregateDevice_<pid>) new in 9.0.0, and is there a supported way to avoid it, or to keep the launch path off synchronous property queries? I am pinned to 8.0.13 purely because of this and I would rather be on 9.

If a diagnostic build with extra logging in the new CoreAudio layer would help, send it over and I will run it and report back the same day. Same for any other test you want.

5

Following up: I tested JUCE 9.0.1, which contains @anthony-nicholls’ fix 0db10a8974 (“CoreAudio: Avoid calls to AudioObjectHasProperty”), and the deadlock is still there on this machine. Same function, same terminal frames. I think the fix is aimed one layer too high, and the stack shows where it needs to move.

Same setup as before: macOS 13.7.8, Intel Mac Pro (Xeon W-3235), Apogee Symphony I/O Mk II over Thunderbolt, 19 CoreAudio devices, Apogee DEXT driver 1.0.20 with the old kext still registered alongside it. JUCE at tag 9.0.1, zero source changes, x86_64 Release. It compiles clean with zero warnings and my full DSP test suite passes 100% against 9.0.1, so the problem is only the live device layer.

The app never reaches its event loop. I have a launch watchdog on a background thread, so it keeps running while the message thread is blocked, and it reported the app still starting at 15.4 s, 31.4 s and 46.7 s with no window. sample puts 100% of 4027 samples on the main thread in one stack:

juce::CoreAudioClasses::CoreAudioIODevice::getInputLatencyInSamples()  + 217
  AudioObjectGetPropertyDataSize                       (in CoreAudio) + 137
    HALPlugIn::ObjectHasProperty (HALObject const&, AudioObjectPropertyAddress const&) const + 36
      HAL_HardwarePlugIn_ObjectHasProperty (...) + 429
        HALC_ShellObject::HasProperty (...) const + 692
          HALC_ProxyObject::HasProperty (...) const + 147
            mach_msg -> mach_msg_overwrite -> mach_msg2_internal -> mach_msg2_trap

The fix’s commit message says to rely on Get/IsProperty* failures instead of AudioObjectHasProperty, and getInputLatencyInSamples now goes through AudioObjectGetPropertyDataSize. But on this device CoreAudio’s own HAL implements AudioObjectGetPropertyDataSize on top of HALPlugIn::ObjectHasProperty, which lands in exactly the same HALC_ProxyObject::HasProperty synchronous round-trip that never returns. The public API changed; the IPC underneath did not. Avoiding AudioObjectHasProperty at the JUCE level is not enough, because the property-existence check still happens inside CoreAudio whichever of the three entry points you call.

That fits everything the July diagnostics showed: patching AudioObject::hasProperty to return true unconditionally got the app past the deadlock, and turning on JUCE_COREAUDIO_LOGGING_ENABLED only moved the hang, to AudioObjectShow in the AudioObject constructor. Same HALC_ProxyObject → mach_msg mechanism, different call site. It was never one specific selector: once the full device list is checked out, this device’s HAL proxy stops answering synchronous property round-trips at all.

From the outside, what looks like it would actually fix it: the latency queries need to tolerate a non-responsive device rather than block the thread that is building the audio device. A timeout, a documented default when the query cannot be satisfied, or moving the latency interrogation off the thread that must return for the app to launch. Any of those turns a hang into a wrong-but-alive latency number, which is recoverable. Blocking indefinitely on mach_msg during device setup is not.

One warning for anyone tempted to reproduce this: force-killing the wedged client wedged coreaudiod itself, after which even my JUCE 8.0.13 build blocked, at CoreAudioIODevice::start() → AudioDeviceStart → mach_msg. Only sudo killall coreaudiod cleared it.

I have also added this to the GitHub issue (#1686). Still happy to run any diagnostic build, a clean stock-JUCE A/B, or to strip the stale Apogee kext if knowing whether the double registration matters would help. The machine is still in this exact state.

6

I think even more so than before this issue should be reported directly to Apogee. As you have access to a device that is failing I think this would be best coming from you. It was one thing to fail on AudioObjectHasProperty but then to have their own implementation of AudioObjectGetPropertyDataSize call HasProperty internally seems like a very serious issue.

When reporting to Apogee I suggest trying to make a small console app that triggers this issue using CoreAudio without JUCE. If you have difficultly putting this together I might be able to help but I will be on holiday for a few weeks now so you might get there quicker with an AI tool to get a basic proof of concept that you can share with them.

AudioObjectGetPropertyDataSize was still being called in the old version so this doesn’t make sense. If patching AudioObject::hasProperty to return true worked then I don’t see why the fix on the develop branch doesn’t work. All that commit does is remove AudioObject::hasProperty.

To be completely clear AudioObject::hasProperty is in the JUCE framework it is not a function called by CoreAudio/HAL. So patching it to return true is almost functionally identical to the provided fix.

In addition to this I had a user on email with the same issue at the same time (I had originally jumped to the conclusion that you were the same person), and the fix included in 9.0.1 seemed like it fixed the issue for them.

Regardless below I’ve supplied a patch that removes calls to AudioObjectGetPropertyDataSize for fixed size properties as strictly speaking it’s not required. Let me know if that works for you.

JUCE-dev-2440e37-CoreAudio: Remove an unnecessary call to AudioObjectGetPropertyDataSize.patch (1.5 KB)

2 Likes
7

Thanks for the patch, and for the correction on hasProperty. Two results and one open question.

The patch does not fix it. Same rig as every report in this thread: Intel Mac Pro (Xeon W-3235),
macOS 13.7.8, Apogee Symphony I/O Mk II over Thunderbolt, the same ~19-device CoreAudio list. I
applied your patch to a scratch JUCE 9.0.1 checkout (our pin stays 8.0.13, this tree is throwaway)
and launched with the Symphony connected. It never reaches the event loop. The stack, patched:
CoreAudioIODevice::getInputLatencyInSamples() → AudioObjectGetPropertyData → HALPlugIn::ObjectHasProperty → HALC_ShellObject::HasProperty → HALC_ProxyObject::HasProperty → mach_msg, never returns.

Control, same session, same rig: I built the identical 9.0.1 tag with your patch reverted and
launched that too, so I am not comparing today against a report from weeks ago. It hangs one call
earlier, reproducing my original trace exactly:
CoreAudioIODevice::getInputLatencyInSamples() → AudioObjectGetPropertyDataSize → HALPlugIn::ObjectHasProperty → HALC_ShellObject::HasProperty → HALC_ProxyObject::HasProperty → mach_msg, never returns.

The only difference between the two traces is which Get-family call sits one frame above
HALC_ProxyObject::HasProperty. Everything below that, three frames, is identical in both. Your patch
removes the AudioObjectGetPropertyDataSize call the same way 0db10a8974 removed
AudioObjectHasProperty, and the result is the same both times: the hang moves one call deeper into
CoreAudio and stops there. It is not fixed, it is relocated.

On the hasProperty correction: you are right, and there is more to it than I had.
AudioObject::hasProperty guarded three separate call sites in the 9.0.0-era code, not the one I was
looking at, so my July reasoning does not survive as stated. But that raises the question rather than
closing it. If AudioObjectGetPropertyDataSize was already in the launch path before your fix the way
you describe, and both Get-family calls independently hit the same wedge on this device, I do not have
an explanation for why forcing hasProperty() to return true got the July build past the deadlock. I
have not read the 9.0.0 to 9.0.1 diff around getInputLatencyInSamples() closely enough to answer
that, and I am not going to guess at it here.

What I think this is worth to you: the fault is not in which entry point JUCE calls to check the
property. Both entry points, forced through independently, land in the identical
HALC_ShellObject::HasProperty → HALC_ProxyObject::HasProperty → mach_msg round trip. That is a
synchronous IPC call into the HAL plug-in that never answers on this device, regardless of which public
API triggers it. Nothing JUCE calls at the CoreAudio layer changes that outcome. That supports your
read that this belongs with Apogee, not you, and gives them something exact to point at rather than
“our app hangs.”

I am building the small CoreAudio-only console reproducer you suggested, no JUCE in it, so I can hand
Apogee something that isolates the proxy round trip on its own. Not built yet. I will take it to them
once I have confirmed it reproduces the hang stripped down.

One more thing worth knowing if you ever chase this yourself: killing the hung test client during this
run wedged coreaudiod for the whole machine, not just our process. An ordinary 8.0.13 build failed
to open the same device immediately afterward, and recovering needed killall coreaudiod, no reboot.
Whatever is stuck on the other end of that mach_msg is stuck for the proxy, not just for one caller.

No rush on your end. I will post again once the reproducer exists.

8

Can you go back and confirm:

  1. JUCE 8.0.13 still works for you
  2. JUCE 9.0.0 with your original hasProperty() hack still works for you

Based on what you’ve shared so far, I’m not convinced we have a clear picture of what works and what doesn’t, so I’d like to confirm those two cases again.

Have you restarted your machine in between any of these tests? If not, can you restart and try checking everything again please. Maybe restart/power-cycle the Apogee device too?

The reason I’m asking is that if AudioObjectGetPropertyData() really does hang independently, then I can’t see a way for JUCE to avoid the problem while still querying the property. In particular, we can’t safely use AudioObjectHasProperty() or AudioObjectGetPropertyDataSize() to determine whether it’s valid to call AudioObjectGetPropertyData(), because those appear to hit the same underlying problem.

But before concluding that, I’d really like to establish whether the original JUCE 9.0.0 workaround still works after a clean restart, and whether 8.0.13 still works under exactly the same conditions.