Different Multichannel Formats with Reaper VST3

Audio Plugins
1

This is somewhat related to another thread (JUCE channel mapping - JUCE) but since disabling the JUCE remapping isn’t possible I was wondering how other people are handling different in/out layouts, particularly in Reaper.

(I am using JUCE 7 and the latest version of Reaper)

Say I make a plugin and I want to support 7.1.4 input with the choice of 7.1.4 and 2nd order Ambisonics (2OA) outputs. In the constructor I define the default bus in/out as 7.1.4:

     : AudioProcessor (BusesProperties()
                       .withInput  ("Input",  juce::AudioChannelSet::create7point1point4(), true)
                       .withOutput ("Output", juce::AudioChannelSet::create7point1point4(), true)
                       )

I then allow say which formats I’ll support in isBusesLayoutSupported:

  bool TestMultichannelPluginAudioProcessor::isBusesLayoutSupported (const BusesLayout& layouts) const
  {
    if ((layouts.getMainOutputChannelSet() == juce::AudioChannelSet::ambisonic(2)
        || layouts.getMainOutputChannelSet() == juce::AudioChannelSet::create7point1point4())
        && layouts.getMainInputChannelSet() == juce::AudioChannelSet::create7point1point4())
        return true;

    return false;
  }

If I am using an AAX in Pro Tools I have no problem because I select the track layout and I can specify if I want 7.1.4 or 2OA as the output. Great.

The problem comes in Reaper (and I think Max 8 but I didn’t test). Reaper just loads the plugin on a 12-channel track that the plugin sees as a 7.1.4 track. If the user selects 2OA output in the plug UI then the default behaviour of a JUCE VST3 in Reaper gives the outputs in the wrong order - they’re remapped by the JUCE wrapper as if the output is 7.1.4.

If these are the only formats I want to support, I can create my own remapper for this case by checking the host. But say I want to support every valid multichannel layout I will have to do this for every combination and monitor the JUCE remapping in case anything changes. That’s a lot of maintenance.

So, I was wondering how other people are handling cases like this? Do you just maintain your own remapping tables depending on the DAW?

Is there some way I’ve missed to force Reaper to have different in/out bus layouts so that the JUCE wrapper applies the correct mapping? I guess there is nothing more to be done from the plugin side to tell Reaper what layout it should use.

Thanks!

EDIT:

Maybe what I’m looking for is getBus(false, 0)->setCurrentLayout();? Is this real-time safe so it can be called from processBlock()?

EDIT 2:
Set setCurrentLayout doesn’t seem to be the solution. According to this post it should not be called from a plugin: