Feature suggestion: AudioIODevice::getLatencyDetails()

Hi, I’m interested in exposing information on the components of audio device latency and have added a method for this to my local JUCE repo. I thought I’d share it here in case there was interest in adopting the concept into the API. It really only adds value for CoreAudio at the moment, as I haven’t yet found any extra information for other device types. Because different devices may return different properties, I’ve implemented the return type as a DynamicObject.

To make use of the interface, you can do something like this:

if (auto* device = deviceManager->getCurrentAudioDevice())
{
    latencyDetails = device->getLatencyDetails();
    for (auto prop : latencyDetails.getProperties())
        DBG ("    " + prop.name + " = " + String (static_cast<int> (prop.value)));
}

(Mention to @ed95 as he has been kind enough to respond to my recent posts relating to ASIO devices :wink: )

Here’s the diff:

juce_AudioIODevice.h

@ -277,6 +277,13 @@ public:
        and the callback getting passed this block of data.
    */
    virtual int getInputLatencyInSamples() = 0;
    
    /** Returns details on the device's latency.
     
        Always includes current buffer size, input and output latency components, but may be overridden to
        include other components. E.g. CoreAudio supplies the safety offsets and stream latencies for both
        input and output devices.
     */
    virtual DynamicObject getLatencyDetails();


    //==============================================================================

juce_AudioIODevice.cpp

@@ -30,6 +30,15 @@ AudioIODevice::AudioIODevice (const String& deviceName, const String& deviceType

AudioIODevice::~AudioIODevice() {}

DynamicObject AudioIODevice::getLatencyDetails()
{
    DynamicObject details;
    details.setProperty ("currentBufferSize", getCurrentBufferSizeSamples());
    details.setProperty ("inputLatency", getInputLatencyInSamples());
    details.setProperty ("outputLatency", getOutputLatencyInSamples());
    return details;
}

void AudioIODeviceCallback::audioDeviceError (const String&)    {}
bool AudioIODevice::setAudioPreprocessingEnabled (bool)         { return false; }
bool AudioIODevice::hasControlPanel() const                     { return false; }

juce_mac_CoreAudio.cpp

@@ -353,6 +353,48 @@ public:

        return (int) (latency + safetyOffset);
    }
    
    struct LatencyDetails
    {
        UInt32 deviceLatency;
        UInt32 safetyOffset;
        UInt32 streamLatency;
    };
    
    LatencyDetails getLatencyDetailsFromDevice (AudioObjectPropertyScope scope) const
    {
        UInt32 deviceLatency = 0;
        UInt32 size = sizeof (deviceLatency);
        AudioObjectPropertyAddress pa;
        pa.mElement = kAudioObjectPropertyElementMaster;
        pa.mSelector = kAudioDevicePropertyLatency;
        pa.mScope = scope;
        AudioObjectGetPropertyData (deviceID, &pa, 0, nullptr, &size, &deviceLatency);

        UInt32 safetyOffset = 0;
        size = sizeof (safetyOffset);
        pa.mSelector = kAudioDevicePropertySafetyOffset;
        AudioObjectGetPropertyData (deviceID, &pa, 0, nullptr, &size, &safetyOffset);

        // Query stream latency
        UInt32 streamLatency = 0;
        UInt32 numStreams;
        pa.mSelector = kAudioDevicePropertyStreams;
        if (OK(AudioObjectGetPropertyDataSize (deviceID, &pa, 0, nullptr, &numStreams)))
        {
            HeapBlock<AudioStreamID> streams (numStreams);
            size = sizeof (AudioStreamID*);
            if (OK(AudioObjectGetPropertyData (deviceID, &pa, 0, nullptr, &size, streams)))
            {
                pa.mSelector = kAudioStreamPropertyLatency;
                size = sizeof (streamLatency);
                // We could check all streams for the device, but it only ever seems to return the stream latency on the first stream
                AudioObjectGetPropertyData (streams[0], &pa, 0, nullptr, &size, &streamLatency);
            }
        }

        return { deviceLatency, safetyOffset, streamLatency };
    }

    int getBitDepthFromDevice (AudioObjectPropertyScope scope) const
    {

@ -415,6 +457,9 @@ public:

        auto newInputLatency  = getLatencyFromDevice (kAudioDevicePropertyScopeInput);
        auto newOutputLatency = getLatencyFromDevice (kAudioDevicePropertyScopeOutput);
        
        auto newInputLatencyDetails = getLatencyDetailsFromDevice(kAudioDevicePropertyScopeInput);
        auto newOutputLatencyDetails = getLatencyDetailsFromDevice(kAudioDevicePropertyScopeOutput);

        Array<CallbackDetailsForChannel> newInChans, newOutChans;
        auto newInNames  = isInputDevice  ? getChannelInfo (true,  newInChans)  : StringArray();

@ -434,6 +479,9 @@ public:

            inputLatency  = newInputLatency;
            outputLatency = newOutputLatency;
            inputLatencyDetails = newInputLatencyDetails;
            outputLatencyDetails = newOutputLatencyDetails;
            
            bufferSize = newBufferSize;

            sampleRates.swapWith (newSampleRates);

@ -810,6 +858,8 @@ public:
    CoreAudioIODevice& owner;
    int inputLatency  = 0;
    int outputLatency = 0;
    LatencyDetails inputLatencyDetails;
    LatencyDetails outputLatencyDetails;
    int bitDepth = 32;
    int xruns = 0;
    BigInteger activeInputChans, activeOutputChans;

@ -1046,6 +1096,21 @@ public:
    {
        return internal->inputLatency;
    }
    
    DynamicObject getLatencyDetails() override
    {
        DynamicObject details = AudioIODevice::getLatencyDetails();
        details.setProperty ("inputDeviceLatency", static_cast<int> (internal->inputLatencyDetails.deviceLatency));
        details.setProperty ("inputSafetyOffset", static_cast<int> (internal->inputLatencyDetails.safetyOffset));
        details.setProperty ("inputStreamLatency", static_cast<int> (internal->inputLatencyDetails.streamLatency));
        details.setProperty ("outputDeviceLatency", static_cast<int> (internal->outputLatencyDetails.deviceLatency));
        details.setProperty ("outputSafetyOffset", static_cast<int> (internal->outputLatencyDetails.safetyOffset));
        details.setProperty ("outputStreamLatency", static_cast<int> (internal->outputLatencyDetails.streamLatency));
        return details;
    }

    void start (AudioIODeviceCallback* callback) override
    {

@ -1518,6 +1583,18 @@ public:

        return lat + currentBufferSize * 2;
    }
    
    DynamicObject getLatencyDetails() override
    {
        DynamicObject details;
        for (auto* d : devices)
        {
            const auto props = d->device->getLatencyDetails().getProperties();
            for (const auto& prop : props)
                details.setProperty (prop.name, jmax (details.getProperty (prop.name), prop.value));
        }
        return details;
    }

    void start (AudioIODeviceCallback* newCallback) override
    {

Was it a mistake to post this under feature requests? :wink:

I’m not sure why no JUCE developers have responded, but I would assume this is the correct place based on the Contributing section in the README.

I have been meaning to respond to this post for while as we recently discovered what seems to be an issue in the latency calculation in the coreaudio device/backend (which I need to get around to creating a bug report for) and your post is somewhat related.

My first question would be why do you want this API and what are you using it for?

I also think that such an API should be able to query the latency per channel even if that isn’t used by some/most backends.

Hi Tim - I’m using it for a new feature I’m adding to RTL Utility. A user has requested that we display the safety offset component.

So my requirement is a bit niche, but I thought I’d see if others were interested. That’d mean I didn’t have to maintain an independent fork of JUCE as I do now.

FWIW, the code changes are implemented on the latencyDetails branch of my fork.

1 Like

Oh ok, sounds potentially useful but why do they want to view the safety offset? Is it because they think the latency is incorrect? or some other reason?

Which sort of leads to my next question: Why do the latency details include components of the I/O latency that CoreAudioInternal::getLatencyFromDevice() does not?

They want to view the safety offset because they are device vendors doing quality control on their device drivers. I want to include it because I’m a bit of a neat freak sometimes :grin:

getLatencyFromDevice() does include the safety offset but my suggestion to also include the stream latency has not been officially implemented.

Now that you’ve pointed it out, I can see I missed putting it in getLatencyFromDevice() in my fork! However, I’m not too fussed because I’ve never seen a non zero value. Fixed now anyway.

Yes, I meant adding the stream latency (which I’ve also yet to see non-zero) but I also meant adding the buffer/frame size, which we found we had to do get correctish roundtrip latency results with the devices we tried with coreaudio. Adding the buffer size also seems consistent with the what is returned by the ASIO and ALSA backends AFAIR so I’m not sure why it isn’t included in the reported latency.

Yep - see here: CoreAudio device latencies are reported inconsistently with other audio device types

Bump and shameless plug - please vote if at all interested!