MacOS Audio Thread Workgroups

I’m writing a multithreaded mac app, its a host app not a plugin, and for performance reasons I really need to make sure my audio processing threads are linked to the device thread, and not just a general high priority thread (its running alot of real time audio processing).

As far as I’m aware the way to do this on MacOs is by using the Audio Workgroups feature, from what I can tell you simply query the device driver for the os_workgroup object using the kAudioDevicePropertyIOThreadOSWorkgroup property then ask your thread(s) to join (and then later leave) that workgroup.

I’ve looked through the juce thread code, including the devel branch (as I know the thread priority are being re-written), and as far as I can tell there isnt support for this in the threads, either in master or devel. It looks like it simply tells the OS its real time, not link it to the device driver. I may be in-correct here mind!

I’ve also tried to simply write my own subclass of Thread to simply run the code to join the workgroup (including getting my head round objective c, as I am more of a C++ person), however without the device id of the audio device, which is not exposed by the AudioIODevice interface that isnt possible.

I even looked at getting the projucer to copy the audio device module into my workspace so I could hack/extend the underlying code, but it gets re-written each time I reload the xcode from the projucer!

Whats my best course of action? What am I missing?

TIA

Chris

2 Likes

Just FYI - unfortunately I don’t know the answer -

See also: https://forum.juce.com/t/mac-m1-thread-priority-audio-workgroups/52188

and https://forum.juce.com/t/fr-thread-priority-vs-efficiency-performance-cores/49025

There is a description on how to retrieve the workgroup here:
https://developer.apple.com/documentation/audiotoolbox/workgroup_management/adding_parallel_real-time_threads_to_audio_workgroups

It mentions retrieving directly from the HAL API - I’m not sure if that’s what the code example is doing with 'workgroup = auHal.osWorkgroup; - I’m just not familiar with any of that code or Objective C.

I’m really hoping the support for audio workgroups just gets integrated into JUCE Thread as a generic option when creating a thread (e.g. some kind of ‘try to join/associate to the ‘AudioProcessor’ audio callback workgroup’ flag/option) - or whatever is the better way to do that.
It’s not just standalone hosting apps that can need multi-threaded real-time audio processing quite a few more powerful synths with multiple layers and large polyphony counts can greatly benefit from it - both as plugins or standalone builds.

Fingers crossed from my side that JUCE devs can take it on soon, because I’m also not keen on figuring out the Objective C side of it and how to integrate it with JUCE Thread, and I’m getting customer complaints about load spiking/audio glitching on M silicon systems - especially those with only 2 e-cores (where it seems some audio threads got demoted to e-cores). I also know of one other developer who maintains a popular plugin hosting app based off JUCE that is getting the same issues.

Cheers,
Paul.

Thanks @wavesequencer

I had read the apple guide on this, but its taken a lot more going through both the juce apple code and also the apple sdk to figure it out.

I am at the point now where I have a class that succesfully adds a thread into an a real time thread and then into audio workgroup without having to modify the core juice classes. Its in objective-c /c++ class with a cpp header. It works by passing in the name of the device and then it searches the core audio device list itself to find the device ID then uses that to add into the audio workgroup. I’m still doing quite a bit of work to actually test it properly, and tune all the real time parameters. But I will post if the testing is succesfull. It could be adapted to work in a plugin I expect.

Thanks

Chris

I made a hack (in juce 7.0.2) to add the AudioDeviceCombiner thread to the device Workgroup and it made a big difference. Here’s the code. I had to change CoreAudioIODeviceType to expose the device IDs as public variables so I could retrieve them when opening a device. I just used the output device to get the workgroup. Hopefully it helps.

Beware, its definitely a quick hack and was not done carefully or built on anything other than my mac.

If you need to make a patch to JUCE, I’d recommend forking it and keeping your own patched repo that you can then rebase whenever JUCE gets updated. We have a few custom patches that we manage this way.

static bool setThisThreadToRealtime (uint64 periodMs)
    {
        const auto thread = pthread_self();

#if JUCE_MAC || JUCE_IOS
        mach_timebase_info_data_t timebase;
        mach_timebase_info (&timebase);

        const auto ticksPerMs = ((double) timebase.denom * 1000000.0) / (double) timebase.numer;
        const auto periodTicks = (uint32_t) jmin ((double) std::numeric_limits<uint32_t>::max(), periodMs * ticksPerMs);

        thread_time_constraint_policy_data_t policy;
        policy.period      = periodTicks/2;
        policy.computation = jmin ((uint32_t) 50000, policy.period)/2;
        policy.constraint  = policy.period;
        policy.preemptible = true;

        return thread_policy_set (pthread_mach_thread_np (thread),
                                  THREAD_TIME_CONSTRAINT_POLICY,
                                  (thread_policy_t) &policy,
                                  THREAD_TIME_CONSTRAINT_POLICY_COUNT) == KERN_SUCCESS;

#else
        ignoreUnused (periodMs);
        struct sched_param param;
        param.sched_priority = sched_get_priority_max (SCHED_RR);
        return pthread_setschedparam (thread, SCHED_RR, &param) == 0;
#endif
    }

Then in your thread run function:

    AudioDeviceID anID = 0;
    os_workgroup_join_token_s JoinToken;

    void run() override
    {
        setThisThreadToRealtime (jmax<int> (1000.0 * currentBufferSize
                                            / float(currentSampleRate),1));

        UInt32 Count = sizeof(os_workgroup_t);
        os_workgroup_t pWorkgroup = NULL;

        ::AudioDeviceGetProperty(anID, 0, 0,
                                 kAudioDevicePropertyIOThreadOSWorkgroup, &Count, &pWorkgroup);


        int Result = ::os_workgroup_join(pWorkgroup, &JoinToken);

        auto numSamples = currentBufferSize;

        AudioBuffer<float> buffer (fifos.getNumChannels(), numSamples);
        buffer.clear();

        Array<const float*> inputChans;
        Array<float*> outputChans;

        for (auto* d : devices)
        {
            for (int j = 0; j < d->numInputChans; ++j)   inputChans.add  (buffer.getReadPointer  (d->inputIndex  + j));
            for (int j = 0; j < d->numOutputChans; ++j)  outputChans.add (buffer.getWritePointer (d->outputIndex + j));
        }

        auto numInputChans  = inputChans.size();
        auto numOutputChans = outputChans.size();

        inputChans.add (nullptr);
        outputChans.add (nullptr);

        auto blockSizeMs = jmax (1, (int) (1000 * numSamples / currentSampleRate));

        jassert (numInputChans + numOutputChans == buffer.getNumChannels());

        threadInitialised.signal();

        while (! threadShouldExit())
        {
            readInput (buffer, numSamples, blockSizeMs);

            bool didCallback = true;

            {
                const ScopedLock sl (callbackLock);

                if (callback != nullptr)
                    callback->audioDeviceIOCallbackWithContext ((const float**) inputChans.getRawDataPointer(),
                                                                numInputChans,
                                                                outputChans.getRawDataPointer(),
                                                                numOutputChans,
                                                                numSamples,
                                                                {}); // Can't predict when the next output callback will happen
                else
                    didCallback = false;
            }

            if (didCallback)
            {
                pushOutputData (buffer, numSamples, blockSizeMs);
            }
            else
            {
                for (int i = 0; i < numOutputChans; ++i)
                    FloatVectorOperations::clear (outputChans[i], numSamples);

                reset();
            }
        }
        os_workgroup_leave(pWorkgroup, &JoinToken);
    }
3 Likes

@blackhawkbravo1 I really hope JUCE does add this capability because this is the exact reason why we use JUCE - so we don’t have to deal with the device level stuff. For our purposes is was only a problem in the AudioDeviceCombiner which is fixed on the develop branch.

Thanks for that @tlacael. Apologies for the long post but I cant upload attachments as I am a new user, but here is my code, which you can add to any existing juce Thread, pass it the name of the device and get yourself added to the workgroup. I’m not convinved I have the real time maths quite right yet - so I might go through yours and see if you have it better sorted.

WMMacAudioThread.h

/*
  ==============================================================================

    WMMacAudioThread
    Created: 2 Feb 2020 9:03:51pm
    Author:  BlackhawkBravo1 (Chris)

  ==============================================================================
*/
#pragma once
//#include "AppConfig.h"
//#include "../JuceLibraryCode/modules/juce_core/juce_core.h"



class WMMacAudioThread {
public:
    WMMacAudioThread();
    ~WMMacAudioThread();
    struct WorkGroup;
    struct JoinToken;
    
    void joinDeviceWorkgroup(const char * DeviceName);
    
    /**Leave the workgroup previously joined, this must be called from the run function before the thread exists**/
    void leaveDeviceWorkGroup();
    
    /**Set the thread to realtime, must be called from within the Run class of the thread before joinDeviceWorkgroup**/
    void setRealTime(int sampleRate, int blockSize);
    
private:
    WorkGroup * workgroup;
    JoinToken * joinToken;
};

WMMacAudioThread.mm

#include "WMMacAudioThread.h"
#import <AudioToolbox/AudioToolbox.h>

#include "AppConfig.h"
#include "../JuceLibraryCode/modules/juce_core/juce_core.h"
#include <mach/mach.h>
#include <mach/mach_time.h>
#include <pthread.h>

struct WMMacAudioThread::WorkGroup { os_workgroup_t _Nonnull data;};
struct WMMacAudioThread::JoinToken { os_workgroup_join_token_s data;};

WMMacAudioThread::WMMacAudioThread(){
    workgroup = new WorkGroup;
    joinToken = new JoinToken;
    
}

WMMacAudioThread::~WMMacAudioThread(){
    delete joinToken;
    delete workgroup;
}

void WMMacAudioThread::setRealTime(int sampleRate, int blockSize){
    
    mach_timebase_info_data_t timebase_info;
    mach_timebase_info(&timebase_info);
    
    const uint64_t NANOS_PER_MSEC = 1000000ULL;
    double clock2abs = ((double)timebase_info.denom / (double)timebase_info.numer) * NANOS_PER_MSEC;
    
 //   thread_port_t threadport = pthread_mach_thread_np(pthread_self());
    
    double periodLength = ((1000.0/(double)sampleRate) * (double)blockSize);
    
    thread_time_constraint_policy_data_t policy;
    policy.period      = (uint32_t)(periodLength * clock2abs);
    policy.computation = (uint32_t)(periodLength/3.0 * clock2abs);
    policy.constraint  = (uint32_t)(periodLength/2.0* clock2abs);
    policy.preemptible = TRUE;
    
    int kr = thread_policy_set(pthread_mach_thread_np(pthread_self()),
                               THREAD_TIME_CONSTRAINT_POLICY,
                               (thread_policy_t)&policy,
                               THREAD_TIME_CONSTRAINT_POLICY_COUNT);
    //  kern_re
    if (kr != KERN_SUCCESS) {
        juce::Logger::writeToLog("Set Realtime failed:"+juce::String( kr));
        // exit(1);
    }
}

void WMMacAudioThread::joinDeviceWorkgroup(const char * name)
{
    juce::String deviceName = juce::String(name);
    
    //create device ID objects for us to fill when we fine the device
    int deviceID = -1;
    AudioDeviceID devID;
    
    {
        //find the device
        UInt32 size;
        AudioObjectPropertyAddress pa;
        pa.mSelector = kAudioHardwarePropertyDevices;
        pa.mScope = kAudioObjectPropertyScopeWildcard;
        pa.mElement = kAudioObjectPropertyElementMain;
        
        if (AudioObjectGetPropertyDataSize (kAudioObjectSystemObject, &pa, 0, nullptr, &size) == noErr)
        {
            juce::HeapBlock<AudioDeviceID> devs;
            devs.calloc (size, 1);
            
            if (AudioObjectGetPropertyData (kAudioObjectSystemObject, &pa, 0, nullptr, &size, devs) == noErr)
            {
                auto num = (int) size / (int) sizeof (AudioDeviceID);
                
                for (int i = 0; i < num; ++i)
                {
                    char name[1024];
                    size = sizeof (name);
                    pa.mSelector = kAudioDevicePropertyDeviceName;
                    
                    if (AudioObjectGetPropertyData (devs[i], &pa, 0, nullptr, &size, name) == noErr)
                    {
                        auto nameString = juce::String::fromUTF8 (name, (int) strlen (name));
                        if(nameString == deviceName){
                            //we have
                            devID = devs[i];
                            deviceID = num;
                            
                            size = 0;
                            pa.mSelector =kAudioDevicePropertyIOThreadOSWorkgroup;
                            
                            if(AudioObjectGetPropertyDataSize( devs[i], &pa, 0, nullptr, &size) == noErr){
                                //     juce::Logger::writeToLog("Size of Struct:"+juce::String(size));
                                
                                os_workgroup_t _Nonnull wkgroup;
                                
                                if (AudioObjectGetPropertyData (devs[i], &pa, 0, nullptr, &size, &wkgroup) == noErr)
                                {
                                  //  os_workgroup_in
                                    //juce::Logger::writeToLog("Successfully got the workgroup");
                                    if(os_workgroup_testcancel(wkgroup)){
                                        juce::Logger::writeToLog("Workgroup cancelled");
                                    }
                                    //   os_workgroup
                                    os_workgroup_join_token_s jt;
                                    
                                    const int result = os_workgroup_join(wkgroup, &jt);
                                    if (result == 0) {
                                        // Success.
                                        juce::Logger::writeToLog("Success joining workgroup");
                                        workgroup->data = wkgroup;
                                        joinToken->data = jt;
                                        
                                        //os_wor
                                    }
                                    else if (result == EALREADY) {
                                        // The thread is already part of a workgroup that can't be
                                        // nested in the the specified workgroup.
                                        juce::Logger::writeToLog("Error:allready in worgroup");
                                    }
                                    else if (result == EINVAL) {
                                        // The workgroup has been canceled.
                                        juce::Logger::writeToLog("Error:workgroup cancelled");
                                    }
                                }
                            }
                        }
                    }
                }
            }
        }
    }
     
    return;
}

void WMMacAudioThread::leaveDeviceWorkGroup(){
    os_workgroup_leave(workgroup->data, &joinToken->data);
}
2 Likes

Im not clear on how to use your class. Do you just put it as a private member of our Thread-Sub-Class?

You can simply Subclass your thread class from this class and the juice thread class, then from your run method call setRealtime before join workgroup, then remember to leave the workgroup before the leave workgroup. I’m still testing it, and I don’t think I have the right numbers in the set real-time.

Chris

Ah ok, got it. Would it be maybe better to subclass JUCe:Thread directly in your class? Doesn’t seem to make any sense in a class that is not a thread subclass.

Your welcome to try if you like, but you’ll run into includes and namespace issues with the headers, as it’s mixing c++ and objective c, plus the memebrs of the Juce namespace conflicts with parts of the apple audio namespace etc.

Hi, thank you for the example!

I am wondering, why did you set policy.period to periodTicks/2? Shouldn’t it “just” be periodTicks, the time between two audio callbacks?

Edit:

I see that JUCE does set its audio thread to realtime, and there they use the below:


thread_time_constraint_policy_data_t policy;
        policy.period = periodTicks;
        policy.computation = jmin ((uint32_t) 50000, policy.period);
        policy.constraint = policy.period;
        policy.preemptible = true;

But then these aren’t part of workgroups of course.

Thanks for the examples @tlacael

So I managed to get audio workgroups stuff to work for AU plugins/audio threads (see FR: Thread-Priority vs Efficiency/Performance Cores - #27 by wavesequencer), but I’m not sure where I will add the similar appropriate code for standalone builds - in my case I am using the JUCE Thread class (so I don’t need to set up the real-time stuff as it’s handled in ‘startRealtimeThread’ function). I bridge the C++ to objective C++/.mm side of things via callbacks that get registered by calling into PluginProcessor from juce_AU_Wrapper.mm.

I saw that I can retrieve the audio device name in ‘audioDeviceAboutToStart’ function within juce_StandaloneFilterWindow.h (seems like a good place since device can change so workgroup can also change at run-time) - but maybe there is a better place to implement the necessary code. It probably needs to be in a separate objective C class/file to avoid type clashing with JUCE namespace classes - since there’s no equivalent .mm wrapper for standalone.
With the device name, it’s possible to retrieve the associated workgroup following your example above @tlacael.

Any advice folks on the best place to retrieve device name and still be able to call into ‘pluginProcessor’ to set my thread workgroup registration callbacks?

I am also trying to figure where to do the same for VST3.
(Or maybe there’s a better approach entirely.)

Update from my side - Was able to get this working for standalone mode by adding a similar helper class as for my AU plugin example directly in juce_StandaloneFilterWindow.h - with the same callback registrations that my threads can then call into for join/leave workgroup - additional function to get the device workgroup copied from the post above from @blackhawkbravo1 - thanks again.

I’m aware I could probably also get the device name via ‘AudioDeviceManager’ class within my own plugin processor code with the ‘prepareToPlay’ function - as it also gets called from ‘audioDeviceAboutToStart’ - but I had some issues with type clashing with JUCE vs CoreAudio - even within juce_StandaloneFilterWindow.h I had to use #undef Point before the include of the CoreAudio header.
There’s probably a cleaner way to get around this issue… this is my quick hack - any recommendations welcome
juce_StandaloneFilterWindow.h (49.9 KB)
.

Now - does anybody know the correct way/place to retrieve the current audio device name (or directly get the device workgroup) when building a VST3 plugin (couldn’t see an obvious way in the juce_VST3_Wrapper)?

2 Likes

For VST3

There are no API yet

Thanks @otristan - maybe there is a way to query the thread workgroup linkage of the main render call thread (instead of needing to query workgroup via the hardware device name as done in the standalone example here)? However that does assume the DAW also registers its main realtime process thread with the audio output hardware workgroup.
I’ve not looked into that yet though. It might already be an improvement also to just create a new workgroup to at least make sure several threads in a thread-pool get treated at the same priority… just thinking aloud here.

You want to get the workgroup from the DAW so the best is to get it from the DAW.
This is how it’s done in the AU format and this is how it should be done in other format IMHO.
But in the meantime, I don’t know how to get the workgroup from the current thread

1 Like

Each Core Audio device provides a workgroup that other realtime threads can join using the os_workgroup_join_self function. Joining the audio device workgroup tells the system that your app’s realtime threads are working toward the same deadline as the device’s thread. You access the workgroup associated with a device in one of several ways:

Seems like we get the workgroup from the device involved? Thus, no need to involve the DAW, which hopefully is joining the same workgroup …

as you said, hopefully :slight_smile:
Plus if there are multiple audio device, you need to get the one opened by the DAW

Here’s the problem - how do you identify the device that the DAW has selected from a VST3 - currently there appears to be no way, unless you make the user confirm the selected device manually within the plugin GUI.

(With AU the API already provides the device workgroup - so it’s not a problem.
With standalone builds, we already know the selected device as that is selected by the application.)

Were you able to get the device from the DAW?