Offline Rendering AudioProcessorGraph [SOLVED]

#1

Hey All,

I’ve been using the AudioProcessorGraph with no problems and have been loving the class, however I’ve hit some issues today with offline rendering.

It appears this issues stem from the Async methods the graph uses to initialize it’s internals.

When Ableton switches the graph to offline mode and back, it appears the async messages are triggering at the incorrect times or something of the sort.

Has anyone else faced these issues in offline rendering with the audiograph?

My prepare to play looks like such:

void APPAudioProcessor::prepareToPlay (double sampleRate, int samplesPerBlock)
{
    mAudioGraph->setPlayConfigDetails(getMainBusNumInputChannels(),
                                      getMainBusNumOutputChannels(),
                                      sampleRate, samplesPerBlock);
    
    mAudioGraph->prepareToPlay(sampleRate, samplesPerBlock);
    
    mAudioGraph->initializeGraph();
}

And my process block:

void APPAudioProcessor::processBlock (AudioBuffer<float>& buffer, MidiBuffer& midiMessages)
{
    ScopedNoDenormals noDenormals;
    auto totalNumInputChannels  = getTotalNumInputChannels();
    auto totalNumOutputChannels = getTotalNumOutputChannels();

    for (auto i = totalNumInputChannels; i < totalNumOutputChannels; ++i)
        buffer.clear (i, 0, buffer.getNumSamples());
            
    mAudioGraph->updatePlayhead(getPlayHead());
    mAudioGraph->processBlock(buffer, midiMessages);
}

Inside of that initializeGraph functions a few things happen.

The graph is first cleared, then the nodes are added, then parameter pointers are placed into the nodes for access to application parameters, and finally a set of default connections are created for audio playback.

Like I said this has all been working wonderfully except for offline mode.

If I don’t get a crash, I get a pocket of unprocessed audio at the start of the offline render, which sounds as though the graph is waiting for the async update to happen, and it didn’t process the audio at all.

I’ve tried swapping the order of things around a bit but to no success. Thanks for any tips!

Best,

J

0 Likes

#2

See this thread:

Are you using the latest JUCE commit?

Rail

0 Likes

#3

Hey Rail,

Thanks for the response. I do see that I’m on the tip and this code is in here.

I was thinking about this being the issue… I’m going to try a few things to call this update synchronously from my process loop if it’s needed… I don’t truthfully understand why a lot of these things are async

0 Likes

#4

Do we have any idea why we want the graph construction to be triggered async from prepareToPlay? In what case is that beneficial?

0 Likes

#5

The AudioProcessorGraph used to be really slow to initialize back in the day if there was more than a few graph nodes involved. And since prepareToPlay is generally called from the GUI thread, maybe they thought it was useful to make it return quickly to the event loop but do the actual work async. But even that doesn’t completely make sense since the heavy work would have been done in the GUI thread anyway, potentially locking it for a long time…So, it’s just puzzling really. :thinking: Maybe it was done because of some host application that doesn’t call prepareToPlay on the GUI thread…?

0 Likes

#6

The graph with a lot of nodes is still very slow… even with some local mods to defer rebuilding when making multiple changes… adding the double support ironically doubled the rebuild time.

Rail

0 Likes

#7

Hmm yeah… I don’t see what would be so slow in here and I only have about ~30 nodes so I haven’t experienced any slow down… but in any case, I can’t see why we would ever want to let process block be called before we’re ready to play…

I think a couple simple fixes would be to remove the amount of async triggers being called, simple add, remove nodes, create connections, etc, and if a change needs to be made, do it at the start of the process block when we know it’s safe…

This method of swapping things in the background async is causing a load of issues, a huge one being the start of the audio processing in an offline render doesn’t have the audio effects applied at all.

I definitely need some manner to force this to trigger before process block is called, otherwise my non real time exports are having windows of dry signal at the start of the export.

0 Likes

#8

Switching my graph initialization to be before the prepareToPlay appears to have improved the crashes somewhat… but yeah, I’m inclined to almost make a copy of the graph and remove all of this async stuff entirely…

0 Likes

#10

So, I’m able to recreate this bug with the AudioProcessorGraph tutorial from the demos.

Simply place the plugin onto a channel, perhaps with some sort of sample.

The plugin should mute the sample, and then replace the output with the oscillator.

If you place the sample in the timeline view in Ableton, and then select a few seconds at the beginning of the timeline, and then export:

You can hear the sample at the start of the export (while the graph is waiting to handle async update) and then for me it was just a muted output, It should have been a sinetone in my test.

Anyways, this indicates to me that the AudioProcessorGraph is unable to render in offline mode (at least in Ableton where I’m testing)

Would love to work with the JUCE team to get this fixed as I just built a framework based around the audio processor graph :grimacing:

My first instinct is to remove the async messaging and message thread logic.

Best,

J

0 Likes

#11

I just exported in Ableton 9 rendering offline with my Drum VI plug-in and it worked 100% perfectly fine. I use the AudioProcessorGraph and the preset I rendered has 219 nodes with 394 connections.

Rail

0 Likes

#12

Thanks for helping check Rail, super appreciate it.

Is that with the most current JUCE?

The test I did was just in the demo app from the learn section…

Are you clearing & adding your nodes every time prepare to play is called?

0 Likes

#13

Yes - the very tip.

As mentioned, I have modified my local copy of the AudioProcessorGraph… but nothing that would indicate that the official release is broken. If you search the forum you’ll probably see most of the changes I’ve made mentioned in my threads related to the graph.

Rail

0 Likes

#14

My prepareToPlay is:

//==============================================================================
void AudioProcessorGraph::prepareToPlay (double sampleRate, int estimatedSamplesPerBlock)
{
    setRateAndBufferSizeDetails (sampleRate, estimatedSamplesPerBlock);
    clearRenderingSequence();
    
    if (/* isNonRealtime() && */ MessageManager::getInstance()->isThisTheMessageThread())
        handleAsyncUpdate();
    else
        triggerAsyncUpdate();
}

in buildRenderingSequence I reduce overhead by making it only build for float…

void AudioProcessorGraph::buildRenderingSequence()
{
    const bool bFloatOnly = true;     // Rail Jon Rogut -- to reduce overhead where I only use float in the graph
    
    if (bFloatOnly)
        {
        std::unique_ptr<RenderSequenceFloat>  newSequenceF (new RenderSequenceFloat());
        
            {
            const ScopedLock sl (buildLock); // MessageManagerLock mml;
            
            RenderSequenceBuilder<RenderSequenceFloat>  builderF (*this, *newSequenceF);
            }
        
            {
            const ScopedLock sl (getCallbackLock());
            newSequenceF->prepareBuffers (getBlockSize());
            }
        
        if (anyNodesNeedPreparing())
            {
                {
                const ScopedLock sl (getCallbackLock());
                renderSequenceFloat.reset();
                }
            
            for (auto* node : nodes)
                node->prepare (getSampleRate(), getBlockSize(), this, getProcessingPrecision());
            }
        
        const ScopedLock sl (getCallbackLock());
        
        std::swap (renderSequenceFloat, newSequenceF);
        }
    else
        {
        std::unique_ptr<RenderSequenceFloat>  newSequenceF (new RenderSequenceFloat());
        std::unique_ptr<RenderSequenceDouble> newSequenceD (new RenderSequenceDouble());
        
            {
            const ScopedLock sl (buildLock); // MessageManagerLock mml;
            
            RenderSequenceBuilder<RenderSequenceFloat>  builderF (*this, *newSequenceF);
            RenderSequenceBuilder<RenderSequenceDouble> builderD (*this, *newSequenceD);
            }
        
            {
            const ScopedLock sl (getCallbackLock());
            newSequenceF->prepareBuffers (getBlockSize());
            newSequenceD->prepareBuffers (getBlockSize());
            }
        
        if (anyNodesNeedPreparing())
            {
                {
                const ScopedLock sl (getCallbackLock());
                renderSequenceFloat.reset();
                renderSequenceDouble.reset();
                }
            
            for (auto* node : nodes)
                node->prepare (getSampleRate(), getBlockSize(), this, getProcessingPrecision());
            }
        
        const ScopedLock sl (getCallbackLock());
        
        std::swap (renderSequenceFloat, newSequenceF);
        std::swap (renderSequenceDouble, newSequenceD);
        }
    
    cancelPendingUpdate();
    
    sendChangeMessage();
}

Rail

0 Likes

#15

Hmm… Well the experience I’m having over here is that the process block is called multiple times before handleASyncUpdate happens, then sometimes randomly in the midst of prepareToPlay causing crashes…

That’s a nice tip to remove the double though thank you!

0 Likes

#16

Sounds like something is possibly hogging the message thread if you’re not getting the call to handleAsyncUpdate().

Rail

0 Likes

#17

Gotcha, well it could perhaps be on my end, or perhaps a live 10 issue, but I was able to repro it with the app from the learn section so hoping someone from the team will help as well. Thanks Rail!

0 Likes

#18

Hello @Jake_Penn,

This issue looks similar to my post: Minor issue with AudioProcessorGraph when fast turning a plugin on and off. (Found by pluginval).

Maybe you could test your plugin with pluginval also, to check if there is a chance it is a JUCE issue?

Pluginval howto is here: Testing plugins with pluginval

Mateusz

0 Likes

#19

Hey Matt,

This is similar. Perhaps when the JUCE team has some time they can address any plans for fixing the graph to not fail here.

0 Likes

#20

Bump bump. : )

Sorry @t0m & @fabian I really don’t mean to press on it but it’s kind of a mission critical bit. Can I follow up over email. If I propose a solution which keeps the async messaging in place but improves the thread safety, is it something we could discuss integrating into the framework?

I’m caught between working to resolve this with everyone here, or creating a fork. I’d really hate to make a fork because it could make things so complicated for anyone to come onto this project in the future.

0 Likes

#21

Hey @Rail_Jon_Rogut, when you tested that did you have audio rendering start from 0:0:0 in the DAW?

0 Likes