Chaining AudioProcessorGraphs


#1

Does anyone have a simple example of how to hook up the output of one AudioProcessorGraph to a Node in another AudioProcessorGraph?

I have:

AudioProcessorGraph::AudioGraphIOProcessor*     m_subProcIn;
AudioProcessorGraph::AudioGraphIOProcessor*     m_subProcOut;

AudioProcessorGraph::Node::Ptr                  m_subProcInputNode;
AudioProcessorGraph::Node::Ptr                  m_subProcOutNode;
m_subProcIn       = new AudioProcessorGraph::AudioGraphIOProcessor (AudioProcessorGraph::AudioGraphIOProcessor::audioInputNode);
m_subProcOut    = new AudioProcessorGraph::AudioGraphIOProcessor (AudioProcessorGraph::AudioGraphIOProcessor::audioOutputNode);

m_subProcInputNode  = m_pSubGraph->addNode (m_subProcIn);
m_subProcOutNode    = m_pSubGraph->addNode (m_subProcOut);

and I try and connect the output m_subProcOut to my mainGraph…. The issue I believe is because I’m trying to create another Reference Counted node to the same Processor:

// Add connection for output of the last m_pSubGraph processor to the output of the subGraph…

addStereoConnection (m_pSubGraph, m_pLastProcOutNode, m_subProcOutNode);

// Now try and connect the subGraph output to a node in the mainGraph:

m_pOutputNode = m_pMainGraph->addNode (m_subProcOut);

// pMaster is a valid node in m_pMainGraph...

addStereoConnection (m_pMainGraph, m_pOutputNode, pMaster);

This last connection isn’t made….

Thanks,

Rail


Graph of graphs(AudioProcessorGraph)
AudioProcessorGraph && ChangeBroadcaster
Can the AudioProcessorGraph please get some love and attention
#2

The graph itself is a processor so would it work to add the graph as a node instead of it's internal output nodes?


#3

Good idea…

m_pOutputNode = m_pMainGraph->addNode (m_pSubGraph);
    
addStereoConnection (m_pMainGraph, m_pOutputNode, pMaster);

That makes the connection… but the output of the subGraph isn’t being fed into the pMaster Nodes…

subGraph last connection:

Connection[032]: Src: Last sub Processor (011) Channel Index: 0 – Dest: Audio Output (003) Channel Index: 0
Connection[033]: Src: Last sub Processor (011) Channel Index: 1 – Dest: Audio Output (003) Channel Index: 1

mainGraph Master Node connections:

Connection[010]: Src: Audio Graph (077) Channel Index: 0 – Dest: Bus: Master (003) Channel Index: 0
Connection[011]: Src: Audio Graph (077) Channel Index: 1 – Dest: Bus: Master (003) Channel Index: 1
Connection[000]: Src: Bus: Master (003) Channel Index: 0 – Dest: Master Track Volume Processor (196) Channel Index: 0
Connection[001]: Src: Bus: Master (003) Channel Index: 1 – Dest: Master Track Volume Processor (196) Channel Index: 1
Connection[250]: Src: Master Track Volume Processor (196) Channel Index: 0 – Dest: Master Track Meter Processor (197) Channel Index: 0
Connection[251]: Src: Master Track Volume Processor (196) Channel Index: 1 – Dest: Master Track Meter Processor (197) Channel Index: 1
Connection[252]: Src: Master Track Meter Processor (197) Channel Index: 0 – Dest: Audio Output (001) Channel Index: 0
Connection[253]: Src: Master Track Meter Processor (197) Channel Index: 1 – Dest: Audio Output (001) Channel Index: 1

The subGraph Nodes (003) aren’t rendering out the mainGraph… are the mainGraph Nodes (077) connected to the Audio Output of the subGraph (003) ??

Perhaps there’s now some other cause of the nodes not rendering…

Thanks!

Rail


#4

I haven't tried using a graph as a node in another graph so I'm not sure of any particular things but you might want to check the following:

-subGraph still uses an output node (AudioGraphIOProcessor) to process the subGraphs main output buffer

-ensure setPlayConfigDetails and prepareToPlay are being called properly for the subGraph.

-setPlayhead is working for the subGraph

-use a dummy processor to debug/verify the subgraph is indeed processing.  Include usage of the playhead here to verify it's being set properly.  Check processor details like channels and block size and sample rate.

If all those things are in place and working then I think it should be operating as expected.  If it's still not pushing audio to the main graph then perhaps it has something to do with how you're connecting the nodes in the overall graph?

 


#5

Hi Graeme,

Thanks for your input.

I have checked all the above and so far they seem to be fine… but there still seems to be no connection between the subGraph’s output and the mainGraph’s node (as indicated in the debug output in my previous post). I’ve minimized the mainGraph and the subGraph and I have a meter proc. as the last processor for the subGraph and the mainGraph… I see metering in the meter processor for the subGraph, but there’s no metering on the mainGraph.

A little history….

I have this all functioning fine using a single (main) AudioProcessorGraph…. all the connections and nodes work fine… the issue is that I have an unrestricted mixer which means the user may add additional tracks as they wish… the problem is that as you add more tracks and the amount of nodes and connections increases then using a single graph becomes very, very slow… based on the more tracks (nodes/connections) you add to the graph. The more connections/nodes, the slower the addNode/addConnection takes. Once the nodes/connections are made the CPU usage settles down and things are fine. I suspect this is because of the sorted OwnedArray used in AudioProcessorGraph and the the linear check in the addNode() method. My idea was to reduce the number of nodes and connections needed to be sorted by creating a new graph for each track… So far it just seems that adding a graph output to another graph may work if you serialize them… but not if you need to add the output of the subGraph to a particular node in another graph. I am doing more testing to see if I’m correct in this conclusion. I’m creating a new standalone app to profile the AudioProcessorGraph.

Thanks,

Rail


#6

Using Instruments to profile the AudioProcessorGraph… the bottleneck is in ConnectionLookupTable::isAnInputToRecursive()

Instruments Trace

Rail


#7

I’d like to add a reference to a previous thread which discusses this same issue (slow graph manipulations with a lot of nodes/connections)

https://forum.juce.com/t/audioprocessorgraph-slow-manipulations

Cheers,

Rail


#8

There were major performance improvements made as a result of that thread.  It looks like the ConnectionSorter::compareElements is a big bottleneck as well.  From a few profiling tests it appears that if we use a hash value on creation of the connection to represent the comparison weighting of compareElements then we see a huge improvement.  The following is just a naive attempt for testing purpose.

e.g.


//in Connection add member
const long hashId;

//then in constructor
AudioProcessorGraph::Connection::Connection (const uint32 sourceNodeId_, const int sourceChannelIndex_,
                                             const uint32 destNodeId_, const int destChannelIndex_) noexcept
:   hashId ( (sourceNodeId *       100000000) +
             (destNodeId *          10000000) +
             (sourceChannelIndex *   1000000) +
             (destChannelIndex *      100000)
           ),
        sourceNodeId (sourceNodeId_), sourceChannelIndex (sourceChannelIndex_),
        destNodeId (destNodeId_), destChannelIndex (destChannelIndex_)
//then in ConnectionSorter::compareElements
if (first->hashId < second->hashId) return -1;
if (first->hashId > second->hashId) return 1;
return 0;

 

EDIT: In fact I think that hashing attempt is faulty.  Just imagine one that would do it properly!

 

 


#9

hmm, looks like my suggestion isn't working right.  I'll have to do some debugging here..


#10

With an idea from jrlanglois I have a very simple solution…. my test has reduced the AudioProcessorGraph::buildRenderingSequence() time to imperceptible… We’re going to discuss my changes and I’ll post the changes here once we’ve decided on the best implementation. The idea is to simply delay building the Rendering Sequence until all the nodes and connections have been made… so buildRenderingSequence() only gets called once.

Cheers,

Rail


#11

Ok thanks Rail.  I'll look forward to seeing what you guys have come up with!  I have my own version of the graph that only calls buildRenderingSequence once but perhaps you are doing something quite different.


#12

Just FYI, I've attached a pic of my profiling.  This is with ~950 nodes.  As you can see buildRenderingSequence is called once and all the time is stuck in the GraphRenderingOps::RenderingOpSequenceCalculator which boils down to indexOfSorted.

 


#13

Yeah, my initial tests were positive… but when I exceed 500 nodes/connections I’m getting the same bottleneck… so I agree… we need a better sort algorithm.

Will check what you posted.

Thanks,

Rail


#14

BTW, I was watching this MS presentation on Modern C++ the other day and they were discussing using std::vector and the prefetcher to improve array profiling… which made me think of this problem…

http://channel9.msdn.com/Events/Build/2014/2-661

Rail


#15

I've been looking at modifying the ConnectionsLookupTable for implementing getConnectionBetween in the ops calculator.  This seems to do a good job of speeding things up.  This time it's with about 1200 nodes.   getConnectionBetween goes from 6333ms to 133ms.  I've attached a profile pic.

 


#16

If you want to share your mod PM me and I can invite you to my test app repos.

I’m currently looking into trying

instead of the OwnedArray to see if it’s any faster doing a linear search using vector as discussed in the video.

Cheers,

Rail


#17

I don't mind sharing it here.  I've just been trying to figure out how to suggest making the change as my modified graph code is quite different.

Maybe I'll just outline the changes.  Perhaps Jules would be interested in making changes to the stock graph in a way that is suitable and proper.

All I'm really doing is adding the connection to the Entry in the lookup table.

In buildRenderingSequence we have:

const GraphRenderingOps::ConnectionLookupTable table (connections);

We modify ConnectionLookupTable::Entry to include an Array of "source" connections


struct Entry
{
    explicit Entry (const uint32 destNodeId_) noexcept : destNodeId (destNodeId_) {}
    
    const uint32 destNodeId;
    SortedSet<uint32> srcNodes;

    //add this
    Array<const Connection*> srcConnections;
    
    JUCE_DECLARE_NON_COPYABLE (Entry);
};

 

In ConnectionLookupTable constructor we add a line to include the source connection

 
for (int i = 0; i < connections.size(); ++i)
{
    const NamAudioProcessorGraph::Connection* const c = connections.getUnchecked(i);
    
    int index;
    Entry* entry = findEntry (c->destNodeId, index);
    
    if (entry == nullptr)
    {
        entry = new Entry (c->destNodeId);
        entries.insert (index, entry);
    }
    
    entry->srcNodes.add (c->sourceNodeId);

    //add this line to include the connection in the entry
    entry->srcConnections.add(c);
}

 

Then we add public getConnectionBetween to ConnectionLookupTable.  (note I've made this version return bool.  The usage in ops calculator only cares that it exists.  Perhaps "isConnected" or something would be a better name)

 
bool getConnectionBetween (const uint32 sourceNodeId, const uint32 sourceChannelIndex, const uint32 destNodeId, const uint32 destChannelIndex) {
    
    int index;
    const Entry* const entry = findEntry (destNodeId, index);
    
    if (entry != nullptr) {
    
        for (int i = 0; i < entry->srcConnections.size(); ++i) {
            const Connection * c = entry->srcConnections.getUnchecked(i);
            
            if (c->sourceChannelIndex == sourceChannelIndex && c->destChannelIndex == destChannelIndex) {
                return true;
            }
            
        }
        
    }
    
    return false;
    
}

 

Then we modify RenderingOpSequenceCalculator to include a const reference to the table and pass the table as a parameter to the calculator's constructor.  (You'll need to move the ConnectionLookupTable class definition to be before the RenderingOpSequenceCalculator class definition. 

//add private class member 
const ConnectionLookupTable & connectionLookupTable;

//modify constructor to accept table reference
RenderingOpSequenceCalculator (AudioProcessorGraph& graph_,
                               const ConnectionLookupTable & connectionLookupTable_,
                               const Array<void*>& orderedNodes_,
                               Array<void*>& renderingOps)
    : graph (graph_),
      connectionLookupTable(connectionLookupTable_),
      orderedNodes (orderedNodes_),
      totalLatency (0)
{






//

 

Then in RenderingOpSequenceCalculator::isBufferNeededLater we replace graph.getConnectionBetween with connectionLookupTable.getConnectionBetween

 


bool isBufferNeededLater (int stepIndexToSearchFrom,
                          int inputChannelOfIndexToIgnore,
                          const uint32 nodeId,
                          const int outputChanIndex) const
{
    while (stepIndexToSearchFrom < orderedNodes.size())
    {
        const NamAudioProcessorGraph::Node* const node = (const NamAudioProcessorGraph::Node*) orderedNodes.getUnchecked (stepIndexToSearchFrom);
        if (outputChanIndex == NamAudioProcessorGraph::midiChannelIndex)
        {
            if (inputChannelOfIndexToIgnore != NamAudioProcessorGraph::midiChannelIndex
                 &&
                 //this is now using the connectionLookupTable
                 connectionLookupTable.getConnectionBetween (nodeId, NamAudioProcessorGraph::midiChannelIndex,
                                                node->nodeId, NamAudioProcessorGraph::midiChannelIndex)
                
                )
                return true;
        }
        else
        {
            for (int i = 0; i < node->getProcessor()->getNumInputChannels(); ++i)
                if (i != inputChannelOfIndexToIgnore
                     && 
                    //this is now using the connection lookup table
                    connectionLookupTable.getConnectionBetween (nodeId, outputChanIndex,
                                                    node->nodeId, i))
                    return true;
        }
        inputChannelOfIndexToIgnore = -1;
        ++stepIndexToSearchFrom;
    }
    return false;
}
 

 

Finally in buildRenderingSequence we move the table out of the local scope block and pass it to the calculator

//move outside of block
const GraphRenderingOps::ConnectionLookupTable table (connections);

{

    for (int i = 0; i < nodes.size(); ++i)
    {
        Node* const node = nodes.getUnchecked(i);
        node->prepare (getSampleRate(), getBlockSize(), this);
        int j = 0;
        for (; j < orderedNodes.size(); ++j)
            if (table.isAnInputTo (node->nodeId, ((Node*) orderedNodes.getUnchecked(j))->nodeId))
              break;
        orderedNodes.insert (j, node);
    }
}

//pass table to calculator
GraphRenderingOps::RenderingOpSequenceCalculator calculator (*this, table, orderedNodes, newRenderingOps);

 

 


#18

A couple things I missed.  The lookup table's getConnectionBetween should be marked const.  And references to Connection may need to have AudioProcessorGraph:: prepended.  I've been applying the changes to the stock graph and found these.


#19

That was an excellent video link btw, thanks!  I'd be interested to hear how you make out with std::vector.


#20

Hi Graeme,

Thanks - I had to make some minor changes to get it to compile with the tip’s AudioProcessorGraph… but I’m still seeing very minor change here in profiling…

JUCE stock adding and connecting 1,000 nodes:

Your code adding and connecting 1,000 nodes:

I’m going to attempt to use std::vector for connections and see what happens…

Cheers,

Rail