After needing to spend some time on other tasks, I had a chance to really dig into this. Perhaps @fabian (or someone with knowledge of multibus and how it relates to VST2) can provide some more insight.
The steps run before processing occurs in VST2 in Wavelab seem to be thus:
Plugin is constructed. The plugin provides the number of input channels and output channels it wants by filling the
vstEffect struct (line 290-291 in Juce_VST_Wrapper.cpp).
vstEffect.numInputChannels = maxNumInChannels;
vstEffect.numOutputChannels = maxNumOutChannels;
Let’s say here that the plugin is stereo-in stereo-out and has a stereo sidechain (4 ins 2 outs).
handleSetSpeakerConfiguration() is called by Wavelab, with 2 inputs and 2 outputs, should Wavelab be processing a stereo file. This function sets the main bus layout of the plugin to match this configuration (line 1897), but doesn’t do anything about the sidechain bus (i.e. leaves it in the initial format).
Upon playback, Wavelab calls
resume() and these lines are run (line 549):
auto numInAndOutChannels = static_cast<size_t> (vstEffect.numInputChannels + vstEffect.numOutputChannels);
floatTempBuffers .channels.calloc (numInAndOutChannels);
The problem is, if you break here,
vstEffect.numOutputChannels are both now equal to 2 (Wavelab changed them!), and
floatTempBuffers is sized accordingly. The plugin, however, is still expecting 4 in channels and 2 out channels, because the bus configuration hasn’t been updated.
When processing is run (
internalProcessReplacing()), the channel counts for iterating the passed
outputs buffers are determined using the plugin’s bus layout (line 398):
const int numIn = processor->getTotalNumInputChannels();
const int numOut = processor->getTotalNumOutputChannels();
So when we iterate over any additional inputs (line 455):
for (; i < numIn; ++i)
tmpBuffers.channels[i] = inputs[i];
tmpBuffers.channels[i] (which is a reference to
floatTempBuffers) will read out of bounds, because it is only sized for 2 ins and 2 outs, and
numIn is 4!
Now, HERE’S THE QUESTION: I’m not sure if this is a bug in Wavelab, or a bug in the JUCE code. It seems that the JUCE code assumes the sidechain will just always be after the main bus inputs in the passed
inputs buffer, but maybe that’s not an assumption that can always be made? Or is Wavelab changing
vstEffect.numOutputChannels when it shouldn’t be? Should a VST2 host always pass buffers to
internalProcessReplacing() that are sized according to the number of channels initially reported by the plugin, or only what was set in
I’ve reached out to Steinberg for more information about this, but would appreciate any further input about what the expected behavior should be with additional input and output buses (e.g. sidechain or aux output) in VST2. It would also be good to perhaps safeguard against this, maybe by checking for
vstEffect.numOutputChannels being different than the bus channel counts in
resume(), and disabling sidechain/aux buses in the processor their if need be?