MacOS Audio Thread Workgroups

I’m writing a multithreaded mac app, its a host app not a plugin, and for performance reasons I really need to make sure my audio processing threads are linked to the device thread, and not just a general high priority thread (its running alot of real time audio processing).

As far as I’m aware the way to do this on MacOs is by using the Audio Workgroups feature, from what I can tell you simply query the device driver for the os_workgroup object using the kAudioDevicePropertyIOThreadOSWorkgroup property then ask your thread(s) to join (and then later leave) that workgroup.

I’ve looked through the juce thread code, including the devel branch (as I know the thread priority are being re-written), and as far as I can tell there isnt support for this in the threads, either in master or devel. It looks like it simply tells the OS its real time, not link it to the device driver. I may be in-correct here mind!

I’ve also tried to simply write my own subclass of Thread to simply run the code to join the workgroup (including getting my head round objective c, as I am more of a C++ person), however without the device id of the audio device, which is not exposed by the AudioIODevice interface that isnt possible.

I even looked at getting the projucer to copy the audio device module into my workspace so I could hack/extend the underlying code, but it gets re-written each time I reload the xcode from the projucer!

Whats my best course of action? What am I missing?

TIA

Chris

1 Like

Just FYI - unfortunately I don’t know the answer -

See also: https://forum.juce.com/t/mac-m1-thread-priority-audio-workgroups/52188

and https://forum.juce.com/t/fr-thread-priority-vs-efficiency-performance-cores/49025

There is a description on how to retrieve the workgroup here:
https://developer.apple.com/documentation/audiotoolbox/workgroup_management/adding_parallel_real-time_threads_to_audio_workgroups

It mentions retrieving directly from the HAL API - I’m not sure if that’s what the code example is doing with 'workgroup = auHal.osWorkgroup; - I’m just not familiar with any of that code or Objective C.

I’m really hoping the support for audio workgroups just gets integrated into JUCE Thread as a generic option when creating a thread (e.g. some kind of ‘try to join/associate to the ‘AudioProcessor’ audio callback workgroup’ flag/option) - or whatever is the better way to do that.
It’s not just standalone hosting apps that can need multi-threaded real-time audio processing quite a few more powerful synths with multiple layers and large polyphony counts can greatly benefit from it - both as plugins or standalone builds.

Fingers crossed from my side that JUCE devs can take it on soon, because I’m also not keen on figuring out the Objective C side of it and how to integrate it with JUCE Thread, and I’m getting customer complaints about load spiking/audio glitching on M silicon systems - especially those with only 2 e-cores (where it seems some audio threads got demoted to e-cores). I also know of one other developer who maintains a popular plugin hosting app based off JUCE that is getting the same issues.

Cheers,
Paul.

Thanks @wavesequencer

I had read the apple guide on this, but its taken a lot more going through both the juce apple code and also the apple sdk to figure it out.

I am at the point now where I have a class that succesfully adds a thread into an a real time thread and then into audio workgroup without having to modify the core juice classes. Its in objective-c /c++ class with a cpp header. It works by passing in the name of the device and then it searches the core audio device list itself to find the device ID then uses that to add into the audio workgroup. I’m still doing quite a bit of work to actually test it properly, and tune all the real time parameters. But I will post if the testing is succesfull. It could be adapted to work in a plugin I expect.

Thanks

Chris

I made a hack (in juce 7.0.2) to add the AudioDeviceCombiner thread to the device Workgroup and it made a big difference. Here’s the code. I had to change CoreAudioIODeviceType to expose the device IDs as public variables so I could retrieve them when opening a device. I just used the output device to get the workgroup. Hopefully it helps.

Beware, its definitely a quick hack and was not done carefully or built on anything other than my mac.

If you need to make a patch to JUCE, I’d recommend forking it and keeping your own patched repo that you can then rebase whenever JUCE gets updated. We have a few custom patches that we manage this way.

static bool setThisThreadToRealtime (uint64 periodMs)
    {
        const auto thread = pthread_self();

#if JUCE_MAC || JUCE_IOS
        mach_timebase_info_data_t timebase;
        mach_timebase_info (&timebase);

        const auto ticksPerMs = ((double) timebase.denom * 1000000.0) / (double) timebase.numer;
        const auto periodTicks = (uint32_t) jmin ((double) std::numeric_limits<uint32_t>::max(), periodMs * ticksPerMs);

        thread_time_constraint_policy_data_t policy;
        policy.period      = periodTicks/2;
        policy.computation = jmin ((uint32_t) 50000, policy.period)/2;
        policy.constraint  = policy.period;
        policy.preemptible = true;

        return thread_policy_set (pthread_mach_thread_np (thread),
                                  THREAD_TIME_CONSTRAINT_POLICY,
                                  (thread_policy_t) &policy,
                                  THREAD_TIME_CONSTRAINT_POLICY_COUNT) == KERN_SUCCESS;

#else
        ignoreUnused (periodMs);
        struct sched_param param;
        param.sched_priority = sched_get_priority_max (SCHED_RR);
        return pthread_setschedparam (thread, SCHED_RR, &param) == 0;
#endif
    }

Then in your thread run function:

    AudioDeviceID anID = 0;
    os_workgroup_join_token_s JoinToken;

    void run() override
    {
        setThisThreadToRealtime (jmax<int> (1000.0 * currentBufferSize
                                            / float(currentSampleRate),1));

        UInt32 Count = sizeof(os_workgroup_t);
        os_workgroup_t pWorkgroup = NULL;

        ::AudioDeviceGetProperty(anID, 0, 0,
                                 kAudioDevicePropertyIOThreadOSWorkgroup, &Count, &pWorkgroup);


        int Result = ::os_workgroup_join(pWorkgroup, &JoinToken);

        auto numSamples = currentBufferSize;

        AudioBuffer<float> buffer (fifos.getNumChannels(), numSamples);
        buffer.clear();

        Array<const float*> inputChans;
        Array<float*> outputChans;

        for (auto* d : devices)
        {
            for (int j = 0; j < d->numInputChans; ++j)   inputChans.add  (buffer.getReadPointer  (d->inputIndex  + j));
            for (int j = 0; j < d->numOutputChans; ++j)  outputChans.add (buffer.getWritePointer (d->outputIndex + j));
        }

        auto numInputChans  = inputChans.size();
        auto numOutputChans = outputChans.size();

        inputChans.add (nullptr);
        outputChans.add (nullptr);

        auto blockSizeMs = jmax (1, (int) (1000 * numSamples / currentSampleRate));

        jassert (numInputChans + numOutputChans == buffer.getNumChannels());

        threadInitialised.signal();

        while (! threadShouldExit())
        {
            readInput (buffer, numSamples, blockSizeMs);

            bool didCallback = true;

            {
                const ScopedLock sl (callbackLock);

                if (callback != nullptr)
                    callback->audioDeviceIOCallbackWithContext ((const float**) inputChans.getRawDataPointer(),
                                                                numInputChans,
                                                                outputChans.getRawDataPointer(),
                                                                numOutputChans,
                                                                numSamples,
                                                                {}); // Can't predict when the next output callback will happen
                else
                    didCallback = false;
            }

            if (didCallback)
            {
                pushOutputData (buffer, numSamples, blockSizeMs);
            }
            else
            {
                for (int i = 0; i < numOutputChans; ++i)
                    FloatVectorOperations::clear (outputChans[i], numSamples);

                reset();
            }
        }
        os_workgroup_leave(pWorkgroup, &JoinToken);
    }
1 Like

@blackhawkbravo1 I really hope JUCE does add this capability because this is the exact reason why we use JUCE - so we don’t have to deal with the device level stuff. For our purposes is was only a problem in the AudioDeviceCombiner which is fixed on the develop branch.

Thanks for that @tlacael. Apologies for the long post but I cant upload attachments as I am a new user, but here is my code, which you can add to any existing juce Thread, pass it the name of the device and get yourself added to the workgroup. I’m not convinved I have the real time maths quite right yet - so I might go through yours and see if you have it better sorted.

WMMacAudioThread.h

/*
  ==============================================================================

    WMMacAudioThread
    Created: 2 Feb 2020 9:03:51pm
    Author:  BlackhawkBravo1 (Chris)

  ==============================================================================
*/
#pragma once
//#include "AppConfig.h"
//#include "../JuceLibraryCode/modules/juce_core/juce_core.h"



class WMMacAudioThread {
public:
    WMMacAudioThread();
    ~WMMacAudioThread();
    struct WorkGroup;
    struct JoinToken;
    
    void joinDeviceWorkgroup(const char * DeviceName);
    
    /**Leave the workgroup previously joined, this must be called from the run function before the thread exists**/
    void leaveDeviceWorkGroup();
    
    /**Set the thread to realtime, must be called from within the Run class of the thread before joinDeviceWorkgroup**/
    void setRealTime(int sampleRate, int blockSize);
    
private:
    WorkGroup * workgroup;
    JoinToken * joinToken;
};

WMMacAudioThread.mm

#include "WMMacAudioThread.h"
#import <AudioToolbox/AudioToolbox.h>

#include "AppConfig.h"
#include "../JuceLibraryCode/modules/juce_core/juce_core.h"
#include <mach/mach.h>
#include <mach/mach_time.h>
#include <pthread.h>

struct WMMacAudioThread::WorkGroup { os_workgroup_t _Nonnull data;};
struct WMMacAudioThread::JoinToken { os_workgroup_join_token_s data;};

WMMacAudioThread::WMMacAudioThread(){
    workgroup = new WorkGroup;
    joinToken = new JoinToken;
    
}

WMMacAudioThread::~WMMacAudioThread(){
    delete joinToken;
    delete workgroup;
}

void WMMacAudioThread::setRealTime(int sampleRate, int blockSize){
    
    mach_timebase_info_data_t timebase_info;
    mach_timebase_info(&timebase_info);
    
    const uint64_t NANOS_PER_MSEC = 1000000ULL;
    double clock2abs = ((double)timebase_info.denom / (double)timebase_info.numer) * NANOS_PER_MSEC;
    
 //   thread_port_t threadport = pthread_mach_thread_np(pthread_self());
    
    double periodLength = ((1000.0/(double)sampleRate) * (double)blockSize);
    
    thread_time_constraint_policy_data_t policy;
    policy.period      = (uint32_t)(periodLength * clock2abs);
    policy.computation = (uint32_t)(periodLength/3.0 * clock2abs);
    policy.constraint  = (uint32_t)(periodLength/2.0* clock2abs);
    policy.preemptible = TRUE;
    
    int kr = thread_policy_set(pthread_mach_thread_np(pthread_self()),
                               THREAD_TIME_CONSTRAINT_POLICY,
                               (thread_policy_t)&policy,
                               THREAD_TIME_CONSTRAINT_POLICY_COUNT);
    //  kern_re
    if (kr != KERN_SUCCESS) {
        juce::Logger::writeToLog("Set Realtime failed:"+juce::String( kr));
        // exit(1);
    }
}

void WMMacAudioThread::joinDeviceWorkgroup(const char * name)
{
    juce::String deviceName = juce::String(name);
    
    //create device ID objects for us to fill when we fine the device
    int deviceID = -1;
    AudioDeviceID devID;
    
    {
        //find the device
        UInt32 size;
        AudioObjectPropertyAddress pa;
        pa.mSelector = kAudioHardwarePropertyDevices;
        pa.mScope = kAudioObjectPropertyScopeWildcard;
        pa.mElement = kAudioObjectPropertyElementMain;
        
        if (AudioObjectGetPropertyDataSize (kAudioObjectSystemObject, &pa, 0, nullptr, &size) == noErr)
        {
            juce::HeapBlock<AudioDeviceID> devs;
            devs.calloc (size, 1);
            
            if (AudioObjectGetPropertyData (kAudioObjectSystemObject, &pa, 0, nullptr, &size, devs) == noErr)
            {
                auto num = (int) size / (int) sizeof (AudioDeviceID);
                
                for (int i = 0; i < num; ++i)
                {
                    char name[1024];
                    size = sizeof (name);
                    pa.mSelector = kAudioDevicePropertyDeviceName;
                    
                    if (AudioObjectGetPropertyData (devs[i], &pa, 0, nullptr, &size, name) == noErr)
                    {
                        auto nameString = juce::String::fromUTF8 (name, (int) strlen (name));
                        if(nameString == deviceName){
                            //we have
                            devID = devs[i];
                            deviceID = num;
                            
                            size = 0;
                            pa.mSelector =kAudioDevicePropertyIOThreadOSWorkgroup;
                            
                            if(AudioObjectGetPropertyDataSize( devs[i], &pa, 0, nullptr, &size) == noErr){
                                //     juce::Logger::writeToLog("Size of Struct:"+juce::String(size));
                                
                                os_workgroup_t _Nonnull wkgroup;
                                
                                if (AudioObjectGetPropertyData (devs[i], &pa, 0, nullptr, &size, &wkgroup) == noErr)
                                {
                                  //  os_workgroup_in
                                    //juce::Logger::writeToLog("Successfully got the workgroup");
                                    if(os_workgroup_testcancel(wkgroup)){
                                        juce::Logger::writeToLog("Workgroup cancelled");
                                    }
                                    //   os_workgroup
                                    os_workgroup_join_token_s jt;
                                    
                                    const int result = os_workgroup_join(wkgroup, &jt);
                                    if (result == 0) {
                                        // Success.
                                        juce::Logger::writeToLog("Success joining workgroup");
                                        workgroup->data = wkgroup;
                                        joinToken->data = jt;
                                        
                                        //os_wor
                                    }
                                    else if (result == EALREADY) {
                                        // The thread is already part of a workgroup that can't be
                                        // nested in the the specified workgroup.
                                        juce::Logger::writeToLog("Error:allready in worgroup");
                                    }
                                    else if (result == EINVAL) {
                                        // The workgroup has been canceled.
                                        juce::Logger::writeToLog("Error:workgroup cancelled");
                                    }
                                }
                            }
                        }
                    }
                }
            }
        }
    }
     
    return;
}

void WMMacAudioThread::leaveDeviceWorkGroup(){
    os_workgroup_leave(workgroup->data, &joinToken->data);
}
1 Like

Im not clear on how to use your class. Do you just put it as a private member of our Thread-Sub-Class?

You can simply Subclass your thread class from this class and the juice thread class, then from your run method call setRealtime before join workgroup, then remember to leave the workgroup before the leave workgroup. I’m still testing it, and I don’t think I have the right numbers in the set real-time.

Chris

Ah ok, got it. Would it be maybe better to subclass JUCe:Thread directly in your class? Doesn’t seem to make any sense in a class that is not a thread subclass.

Your welcome to try if you like, but you’ll run into includes and namespace issues with the headers, as it’s mixing c++ and objective c, plus the memebrs of the Juce namespace conflicts with parts of the apple audio namespace etc.