Bigger buffer sizes when using Oboe compared to OpenSLES

After upgrading our Android application to JUCE 6 we noticed that the latency increased. It is an instrument sampler application triggered by incoming midi notes. The processing time of the JUCE 5 version of the application is considerably faster.

To investigate I ran JUCE 5 & 6 based DemoRunners to find out that the minimum buffer sizes increased in JUCE 6. This seems to be tied to the use of Oboe in JUCE 6.

After I forced the build to use OpenSLES the old buffer sizes where available again.

  • Is there any good reason to not use OpenSLES?

  • Why are the Oboe buffer sizes bigger than the OpenSLES buffer sizes?

  • Is there a way to get smaller buffer sizes when using Oboe?

When looking int the logs of the Android device I noticed some entries about the AUDIO_OUTPUT_FLAG_FAST (AAudio) being denied:

AudioTrack: AUDIO_OUTPUT_FLAG_FAST denied by client, not shared buffer and transfer = TRANSFER_SYNC

I’m aware of the general audio performance of Android devices.

Thanks!

EDIT: A warning following the one mentioned above tells me that the performance mode is not optimal:
W/AudioStreamTrack: open() perfMode changed from 12 to 10
Which basically means that the performance mode is set back to None instead of LowLatency.

BTW: This also reveals that the logging JUCE produces is inaccurate because JUCE outputs the textual representation of the oboe::PerformanceMode::LowLatency mode without getting the actual performance mode of the stream first.

EDIT: For the sake of completeness:

2020-10-20 16:21:28.650 12860-12860/com.company.androidapp I/JUCE: Preparing Oboe stream with params:
    AAudio supported = 1
    API = Unspecified
    DeviceId = 0
    Direction = Output
    SharingMode = Exclusive
    ChannelCount = 2
    Format = Float
    SampleRate = 48000
    PerformanceMode = LowLatency
2020-10-20 16:21:28.650 12860-12860/com.company.androidapp I/OboeAudio: openStream() OUTPUT -------- OboeVersion1.4.2 --------
2020-10-20 16:21:28.650 12860-12860/com.company.androidapp D/AAudio: AAudioStreamBuilder_openStream() called ----------------------------------------
2020-10-20 16:21:28.650 12860-12860/com.company.androidapp D/AudioStreamBuilder: build() EXCLUSIVE sharing mode not supported. Use SHARED.
2020-10-20 16:21:28.651 12860-12860/com.company.androidapp I/AAudioStream: open() rate   = 48000, channels    = 2, format   = 0, sharing = SH, dir = OUTPUT
2020-10-20 16:21:28.651 12860-12860/com.company.androidapp I/AAudioStream: open() device = 0, sessionId   = 0, perfMode = 12, callback: OFF with frames = 0
2020-10-20 16:21:28.651 12860-12860/com.company.androidapp I/AAudioStream: open() usage  = 1, contentType = 2, inputPreset = 6
2020-10-20 16:21:28.651 12860-12860/com.company.androidapp D/AudioStreamTrack: open(), request notificationFrames = 0, frameCount = 0
2020-10-20 16:21:28.651 12860-12860/com.company.androidapp W/AudioTrack: AUDIO_OUTPUT_FLAG_FAST denied by client, not shared buffer and transfer = TRANSFER_SYNC
2020-10-20 16:21:28.656 12860-12860/com.company.androidapp W/AudioStreamTrack: open() flags changed from 0x00000104 to 0x00000100
2020-10-20 16:21:28.656 12860-12860/com.company.androidapp W/AudioStreamTrack: open() perfMode changed from 12 to 10
2020-10-20 16:21:28.656 12860-12860/com.company.androidapp D/AAudio: AAudioStreamBuilder_openStream() returns 0 = AAUDIO_OK for (0x7d65bfc900) ----------------
2020-10-20 16:21:28.658 12860-12860/com.company.androidapp D/OboeAudio: AudioStreamAAudio.open() format=2, sampleRate=48000, capacity = 1924
2020-10-20 16:21:28.658 12860-12860/com.company.androidapp D/OboeAudio: AudioStreamAAudio.open: AAudioStream_Open() returned AAUDIO_OK
2020-10-20 16:21:28.658 12860-12860/com.company.androidapp I/JUCE: Building Oboe stream with result: OK
    Stream state = Open
2020-10-20 16:21:28.658 12860-12860/com.company.androidapp I/JUCE: Setting the bufferSizeInFrames to 192
2020-10-20 16:21:28.658 12860-12860/com.company.androidapp I/JUCE: Stream details:
    Uses AAudio = 1
    DeviceId = 3
    Direction = Output
    SharingMode = Shared
    ChannelCount = 2
    Format = Float
    SampleRate = 48000
    BufferSizeInFrames = 962
    BufferCapacityInFrames = 1924
    FramesPerBurst = 962
    FramesPerCallback = 0
    BytesPerFrame = 8
    BytesPerSample = 4
    PerformanceMode = None

EDIT: We’re seeing this problem on Nokia 5 and Xiaomi Redmi 8

1 Like

Thanks for the bug report, we’ll get the corrected stream details fix added.

Oboe should choose the best audio path for the device (either using OpenSL or AAudio) so it’s hard to say why you are seeing performance regressions when using it. The Oboe devs are quite responsive on the GitHub repo so you might get more answers by posting an issue there:

Thanks Ed,

I will open an issue there.

To pinpoint the problem I tried running the OboeTester app, and it does behave correctly (ie. low buffer size of 192, low latency).

I also compared JUCE 5 based DemoRunner with a JUCE 6 based DemoRunner. The JUCE 5 version (which uses OpenSLES) give a minimal buffer size of 192 samples whereas the JUCE 6 version starts with a buffer size of 962 samples (which uses Oboe and AAudio).

The difference between OboeTester and the JUCE DemoRunners make me wonder if maybe JUCE has a suboptimal way of opening Oboe streams which prevent from getting low(er) latency streams.

Let me turn this thread into a bug report.

After I got response from Phil on the oboe repository I found out the cause of problem.

The reason why this is happening is because JUCE verifies the result of AndroidHighPerformanceAudioHelpers::getNativeBufferSizeHint() by opening a temporary OboeStream (which uses oboe::AudioStreamBuilder under the hood). This temporary stream however is not given an oboe::AudioStreamCallback which prevents Oboe from opening with the AUDIO_OUTPUT_FLAG_FAST flag. Because of that the underlying API (AAudio in this case) comes back with a buffer size much bigger than the requested ideal one.

To (temporarily) solve this issue I created a dummy callback:

class DummyAudioStreamCallback : public oboe::AudioStreamCallback
{
public:
    oboe::DataCallbackResult onAudioReady(oboe::AudioStream* oboeStream, void* audioData, int32_t numFrames) override {}
};

And then OboeAudioIODevice::getNativeBufferSize becomes:

static int getNativeBufferSize()
{
    auto bufferSizeHint = AndroidHighPerformanceAudioHelpers::getNativeBufferSizeHint();

    DummyAudioStreamCallback dummyAudioStreamCallback;

    // NB: Exclusive mode could be rejected if a device is already opened in that mode, so to get
    //     reliable results, only use this function when a device is closed.
    //     We initially try to open a stream with a buffer size returned from
    //     android.media.property.OUTPUT_FRAMES_PER_BUFFER property, but then we verify the actual
    //     size after the stream is open.
    OboeAudioIODevice::OboeStream tempStream (oboe::kUnspecified,
                                              oboe::Direction::Output,
                                              oboe::SharingMode::Exclusive,
                                              2,
                                              getAndroidSDKVersion() >= 21 ? oboe::AudioFormat::Float : oboe::AudioFormat::I16,
                                              (int) AndroidHighPerformanceAudioHelpers::getNativeSampleRate(),
                                              bufferSizeHint,
                                              &dummyAudioStreamCallback);

    if (auto* nativeStream = tempStream.getNativeStream())
        return nativeStream->getFramesPerBurst();

    return bufferSizeHint;
}

After this change I’m able to request low(er) buffer sizes (192 frames in my case).

2 Likes

Thanks for following up! That seems like a reasonable fix to ensure that we’re getting the best available buffer sizes on all devices. We’ll push something shortly.

Many thanks!