AudioDeviceSelectorComponent - UI locking up for 10 secs on iOS

I’m using the AudioDeviceSelectorComponent in an iPad app.

I’m having problems with UI lock ups when changing audio settings. The same problem affects the AudioSettingsDemo, so this time it’s not my code at fault…

Repro steps:

  • Compile the AudioSettingsDemo for iOS and run it on the device.
  • When the app appears, click the Audio buffer size drop down, and select a different buffer size to the one already selected
  • The item you touch will stay selected for 2-3 seconds, with the UI locked up, before eventually the drop down closes and it becomes usable again.
  • If I repeat the exercise with it plugged into a FocusRite iTrack Dock, the UI locks up for 10s every time

So it looks like this is already a small problem with inbuilt iOS audio, but a much bigger problem with the iTrack Dock connected, making the AudioDeviceSelectorComponent unusable.

This occurs after disconnecting the debugger, and running a Release rather than a Debug build.

I’m using Projucer 5.4.5, Xcode 10.1, iOS 12.4.3, iPad mini 2 (A1489)

Any thoughts on how to debug/workaround appreciated!

i have customers seeing the same issues on ios with an iTrack Dock. when connected to the dock my app startup takes ~30s and triggers an ios watchdog crash. if i start the app without the dock connected and then connect the app then everything slows down and all the audio settings take ~10s to update.

so i set JUCE_IOS_AUDIO_LOGGING and get this output when connected to the dock:

Creating iOS audio device
Updating hardware info
Lowest supported sample rate: 44100
Highest supported sample rate: 96000
Trying a sample rate of 45100, got 48000
Trying a sample rate of 49000, got 88200
Trying a sample rate of 89200, got 96000
Available sample rates: 44100 48000 88200 96000
Sample rate after detecting available sample rates: 44100
Available buffer sizes: 64 128 256 512 1024 2048 4096
Buffer size after detecting available buffer sizes: 256
Input channel configuration: {Number of hardware channels: 2, Hardware channel names: "iTrack Dock 1" "iTrack Dock 2", Are channels available: yes, Active channel indices:, Inactive channel indices: 0 1}
Output channel configuration: {Number of hardware channels: 2, Hardware channel names: "iTrack Dock 1" "iTrack Dock 2", Are channels available: yes, Active channel indices:, Inactive channel indices: 0 1}
Opening audio device: inputChannelsWanted: 0, outputChannelsWanted: 11, targetSampleRate: 44100, targetBufferSize: 512
Input channel configuration: {Number of hardware channels: 2, Hardware channel names: "Left" "Right", Are channels available: yes, Active channel indices:, Inactive channel indices: 0 1}
Output channel configuration: {Number of hardware channels: 2, Hardware channel names: "iTrack Dock 1" "iTrack Dock 2", Are channels available: yes, Active channel indices: 0 1, Inactive channel indices:}
Updating hardware info
Lowest supported sample rate: 44100
Highest supported sample rate: 96000
Trying a sample rate of 45100, got 48000
Trying a sample rate of 49000, got 88200
Trying a sample rate of 89200, got 96000
Available sample rates: 44100 48000 88200 96000
Sample rate after detecting available sample rates: 44100
Available buffer sizes: 64 128 256 512 1024 2048 4096
Buffer size after detecting available buffer sizes: 256
Setting target sample rate: 44100
Actual sample rate: 44100
Setting target buffer size: 512
Actual buffer size: 512
Creating the audio unit
Internal buffer size: 4096

this takes 10-20s in debug mode.

when i disconnect the dock and start the app i get this output:

Creating iOS audio device
Updating hardware info
Lowest supported sample rate: 44100
Highest supported sample rate: 48000
Trying a sample rate of 45100, got 48000
Available sample rates: 44100 48000
Sample rate after detecting available sample rates: 44100
Available buffer sizes: 64 128 256 512 1024 2048 4096
Buffer size after detecting available buffer sizes: 256
Input channel configuration: {Number of hardware channels: 1, Hardware channel names: "iPad Microphone", Are channels available: yes, Active channel indices:, Inactive channel indices: 0}
Output channel configuration: {Number of hardware channels: 2, Hardware channel names: "Speaker 1" "Speaker 2", Are channels available: yes, Active channel indices:, Inactive channel indices: 0 1}
Opening audio device: inputChannelsWanted: 0, outputChannelsWanted: 11, targetSampleRate: 44100, targetBufferSize: 512
Input channel configuration: {Number of hardware channels: 2, Hardware channel names: "Left" "Right", Are channels available: yes, Active channel indices:, Inactive channel indices: 0 1}
Output channel configuration: {Number of hardware channels: 2, Hardware channel names: "Speaker 1" "Speaker 2", Are channels available: yes, Active channel indices: 0 1, Inactive channel indices:}
Updating hardware info
Lowest supported sample rate: 22050
Highest supported sample rate: 48000
Trying a sample rate of 23050, got 24000
Trying a sample rate of 25000, got 32000
Trying a sample rate of 33000, got 44100
Trying a sample rate of 45100, got 48000
Available sample rates: 22050 24000 32000 44100 48000
Sample rate after detecting available sample rates: 44100
Available buffer sizes: 64 128 256 512 1024 2048 4096
Buffer size after detecting available buffer sizes: 256
Setting target sample rate: 44100
Actual sample rate: 44100
Setting target buffer size: 512
Actual buffer size: 512
Creating the audio unit
Internal buffer size: 4096

and this happens almost instantly.

any juce folks have ideas here?

I debugged this a bit, and it looks like the main culprit is the audio device initialisation, which currently happens on the main thread. This can commonly take a second or so, even for the built-in audio devices on a MacBook or iPhone, and the message thread can’t progress during this time.

I’ve put together a patch which attempts to open audio devices on a background thread, in order to free up the message thread and keep the UI responsive.

Unfortunately, I don’t have an iTrack dock for testing, so I’m not sure whether this patch will completely resolve the issue. It’s possible that there are other time-consuming calls which will still happen on the main thread. I’d appreciate if you could test this change (it must apply on top of the develop branch) and check whether it fixes the issue for you.

audiodeviceselector.patch (56.2 KB)

thanks for the patch @reuk. it doesn’t seem to resolve the issue though.

i actually tracked down one of these docks to repro the issue - dm me and i’ll ship it to you if it’ll help sort this out.

I’ve got my hands on an iTrack Dock and debugged a bit with it. As far as I can tell, the main slowdown is due to calling AVAudioSession::setPreferredSampleRate, which takes about two seconds each time. Unfortunately, the only way to query available sample rates on iOS (as far as I can tell) is to set a lot of different potential sample rates, and to query which rates the device managed to apply. Currently, we query lots of sample rates when the AudioDeviceManager is initialised, which is why the main thread can freeze for ~30s while the app opens.

Working around these slow samplerate queries is surprisingly tricky! One option might be to add an option to initialise the AudioDeviceManager asynchronously, which would free up the main thread. However, this approach still wouldn’t be perfect:

  • The audio device would still take a long time to become available. The app would appear to have no audio ins/outs until the initialisation completed.
  • Adding this async functionality would add quite a lot of complexity to the AudioDeviceManager implementation (lots of potential new bugs), which seems a bit risky given that the problem only affects a few devices on a single platform.

I don’t think we can push just the samplerate queries onto a background thread, because they modify the AVAudioSession singleton - changing the preferred samplerate from a background thread might interrupt a playing audio stream.

Another idea I had was to back-out of querying all the device samplerates if any individual call to setPreferredSampleRate took too long, but this still slows down startup (we have to wait through at least one slow call).

I checked out AUM, and it looks like their solution is to provide a fixed list of possible sample rates (44.1K, 48K, 88.2K, 96K), and to display a warning if switching to a different sample rate fails. This seems like quite a reasonable solution, although it does mean that AUM doesn’t necessarily have the option of switching to more unusual sample rates that might be supported on a particular device.

At the moment, I’m leaning towards adding a compile-time option to prefer a fixed set of samplerates on iOS, rather than attempting to query the audio device. In testing, this seems to work well. The app starts quickly when plugged into the dock, and switching samplerates only takes a couple of seconds. This change also seems a lot safer than adding a lot of complicated async machinery to the AudioDeviceManager. I still need to discuss this a bit more with the team, but hopefully I’ll have some more news next week.

In the meantime, please let me know if you don’t think this approach would work for you.

2 Likes

i just got another bug report for another audio interface and i’m gathering info, so i think anything that improves the situation would be welcome.

Thanks for your patience. I’ve added the following commit on develop:

The idea is that a preferred set of samplerates can be specified using the JUCE_IOS_AUDIO_EXPLICIT_SAMPLERATES preprocessor definition. If defined, JUCE will never query the device for its supported rates, and will use the hard-coded list instead.

If you’re using the AudioDeviceSelectorComponent, you should make sure to include the following commit too:

2 Likes

this seems to fix things for me, thanks!

1 Like

HI,
I’m a bit confused
I’d like to have just 44100 and 48000 in my standalones devicemanager.
I’ve got juce 7.0.5 here and your commits above are not included, do I need to apply this myself still or…?

The above commits are included in JUCE 7.0.5.

2 Likes

ok tnx, dont know what caused the confusion here, I’ve updated to 7.0.6 and the commits are there.
One question:
I added the define as such on my side of the code, is this correct or do I need to add namespace or something?
#define JUCE_IOS_AUDIO_EXPLICIT_SAMPLERATES 44100,48000

That definition should work, but the definition will need to be visible to the translation unit that includes juce_Audio_ios.cpp. Therefore, you’ll probably want to set it in the preprocessor flags for your project in the Projucer or CMake.

1 Like

ah ok, I allready was thinking about the order of initialiseation there… will do it in jucer

ok! brilliant, now it works!
A simular thing could/should be added for hiding bleutooth headphones, to disable AVAudioSessionCategoryOptionAllowBluetooth.
But that is a quick hack after every juce update… thanks again!

2 Likes