Adding parameters post launch to a plugin?

[refusesoftware]

@benvining

If you ever foresee changing anything about the host-facing parameter list in future versions of your plugin, then I think maintaining some kind of versioning system (and saving this version information with your plugin state) is absolutely essential.

When using XML created from an APVTS to store plugin state, versioning is always an option, even if you didn’t have the foresight to include an explicit version property/attribute in the first release of your plugin. Since you’ve got to provide a valueTreeType Identifier for the APVTS, you can always modify that Identifier for future versions that might have a different parameter layout. For example. APVTS type might be MyGreatPlugin for the initial release version, but then after you add parameters, switch the APVTS type to MyGreatPlugin_v2.

@benvining

Of course, the ideal is that DAW sessions created with an older version of your plugin and loaded with a new one will “just work”, but if anything ever goes wrong, having version information accessible to you in setStateInformation will probably make any bugs much easier to identify and fix.

Save/load of plugin settings through getStateInformation/setStateInformation can be handled (e.g. translated if necessary) by versioning parameter layouts. The problem remains however that automation data from the host is not versioned, and AFAIK you can still run into trouble there, if the index of parameters gets shuffled around by a certain DAW.

[daniel]

Just be aware that this versioning only works for the saved values in the getStateInformation/setStateInformation.

The recorded automations however will still be mapped by the host to the parameter index or the UID for those plugin formats that support it. That means if the parameter index changed by adding a different parameter after recording that automation, you will likely get the wrong parameter automated.

[reuk]

I’ve put together a potential workaround for this issue:

logic-parameter-workaround.patch (35.0 KB)

The basic idea is to do something similar to how @gavinburke was handling legacy parameter versioning. Each parameter has a ‘version hint’ which is used to sort the parameter IDs before presenting them to logic. That is, we sort by the ID hash (to simulate the old behaviour), and then do a stable-sort by the version hint to ensure that newer parameters are ordered after older parameters.

I’m sharing the patch here, rather than merging directly, so that those interested can test it out and provide feedback on the design.

This workaround should mean that it’s possible to add parameters to an AUv2 plugin between releases. It’s still not possible to remove parameters, because doing so will cause the indices of all following parameters to change.

I also tested the behaviour of AUv3 plugins in Logic, and those seem to have a similar problem (adding parameters can break saved automation). However, at the moment I can’t see any workaround for this problem, as AUv3 plugins can’t control the order of parameters in the flattened list. If I’m wrong about that, and there is an API to control parameter ordering, please let me know so that I can implement a fix for AUv3 plugins! Otherwise, I can only advise that parameters should never be added or removed on an AUv3 plugin after the initial release, if the plugin is to remain compatible with existing Logic projects.

[reuk]

I’ve tested the proposed change with Final Cut Pro 10.6.1 (the latest version) and it seems to recall automation correctly after a parameter is added. This was the case even before applying the parameter-ordering patch, so I think FCP must be using the hashed parameter identifiers correctly. In addition, the version I tested displayed the parameter groups correctly.

Unfortunately I don’t have access to any older versions of FCP. If you do, I’d appreciate any feedback you can offer if you’re able to try out this change.

[unlinked1]

This looks like a good solution, and the API makes sense at first reading. Thanks @reuk, nice one. I should have a chance to run some tests over the next week and will report back.

[jakemumu]

THIS IS SO IDEAL

[unlinked1]

@reuk I ran some tests here now with a 25-ish parameter plugin, which also uses subgroups. I assigned the parameters randomly to one of three versions, and checked the automation saved with “earlier” versions of the plugin was maintained in later versions.

It behaves as expected in Logic as far as I can tell and didn’t affect the existing working behaviour in Live (not that I would have expected it to as Live seems to be doing the right thing with the hashed IDs anyway).

The API design was straightforward to use from my perspective too. In terms of the implementation my only observation is that presumably, if a debug build is asserting if a versionHint of 0 is detected in AudioProcessor::validateParameter, code will almost always have to supply explicit values for not only versionHint but also parameterLabel & parameterCategory when constructing parameters derived from AudioProcessorParameterWithID? If so, having default values for those parameters in the constructor will usually result in a runtime assertion rather than a compile error. I’m not sure whether this is preferable. But either way, this seems good to me.

[reuk]

Thanks for testing that. It sounds like the change is working for you, so I’ll try to get it merged at some point during this week.

re. the default constructor arguments, I’m not sure about the best approach. Forcing users to supply a version hint would break source compatibility, and require all of our users to make changes to their code. Not all users are building AU plugins in the first place, and even those who are might not care about Logic in particular. For that reason, checking at runtime and only when JucePlugin_Build_AU=1 seemed preferable. Having to supply all of the other constructor arguments is a bit annoying though.

One option would be to allow setting the version hint after construction, although allowing it to change at runtime makes me a bit uncomfortable. I suppose we could add a (runtime) check that the parameter hint may only be set prior to adding the parameter to an AudioProcessor. This would allow users to avoid setting otherwise-defaulted parameter attributes.

Another option would be a builder-style constructor, something like this:

const auto attributes = AudioParameterBool::Attributes ("id", "name")
                            .withDefaultValue (true)
                            .withLabel ("label")
                            .withVersionHint (2);
const auto param = std::make_unique<AudioParameterBool> (attributes);

Unfortunately, updating would still be a bit annoying due to the new constructor/method names.

I’ll discuss this a bit with the team too, but at the moment I think I’m leaning towards reverting the parameter constructors, and adding a setVersionHint function instead.

[daniel]

@reuk

Unfortunately I don’t have access to any older versions of FCP. If you do, I’d appreciate any feedback you can offer if you’re able to try out this change.

Sorry for the late reply. The FCP project is on hold. The client took it down because they have no budget for maintenance at the moment. That’s why I didn’t check.
I just wanted to raise awareness. Thank you for taking it into account and doing those tests.

[unlinked1]

Personally, adding a separate setVersionHint feels a bit worse as versionHint is intimately linked to the parameter id and should be as immutable after construction. Maybe just defaulting to 1 or removing the assertion would be fine as it maintains the current behaviour anyway.

I look forward to whatever you and the team settle on though! Thanks again!

[reuk]

The danger about defaulting to ‘1’ is that it doesn’t force users to consider the parameter version every time they add parameters. I want to avoid the situation where a user releases a product, adds a parameter using the default settings, sees no warnings, and releases a new version. Unfortunately this does mean that the user will need to set a version of ‘1’ on all parameters initially - but I think it’s better to err on the side of caution in this case.

[jpo]

Not sure if that is directly relevant to the issue here, but the AU code from Apple does not respect parameter ordering when using param ids, as explained here:

AU Parameter Order

In a plugin I have whose params are not inheriting AudioProcessorParameter the parameters order is the same in AU and VST. But in plugins using AudioProcessorParameterWithID, the order is different in the AU indeed. This is because the JUCE AU Wrapper calls AUScopeElement::UseIndexedParameters (numParams) when you use legacy parameters. Apparently the Apple audiounit code stores the parameter list in a std::map when using paramID instead of indexes. And its GetParameterList implementation is: …

In my opinion it is a bug. By applying the patch I suggest there, I think Logic can handle addition of new params as long a they appended at the end of the list, and not inserted in the middle of the param list – I have not tested this particular case, though, but I’ve been using that patch since then.

[yfede]

Another brainstorming idea for specifying the versionHint:

Since the versionHint is closely related to the parameter ID, lets have a struct like (not really passing those Strings by value in the ctors, it’s just to keep the code short):

struct IdWithVersionHint
{
    IdWithVersionHint (String id, int version) :
    idToUse (id), versionHint (version)
    { }

    IdWithVersionHint (String id) :
    idToUse (id)
    { }

    String idToUse;
    int versionHint = 0;
};

Then replace the id argument taken by every AudioParameter constructor with one of these structs, like:

AudioParameterBool (const IdWithVersionHint& parameterID, const String& parameterName, /* etc. */);

existing code still compiles thanks to the IdWithVersionHint constructor that takes only a String, e.g.

new AudioParameterBool ("id" /* <- implicitly converted to IdWithVesionHint */, 
                        "name", 
                        /* etc. */); 

Only when building for AU, the jasserts will warn the developer if a versionHint was not specified, in which case it will be solved by adding the versionHint to the IdWithVersionHint argument:

new AudioParameterBool ({"id", 1} /* id with versionHint = 1 added */, 
                        "name", 
                        /* etc. */);

[daniel]

I don’t intend to hijack, but when you are about to redesign the AudioParameter interface, please consider a

struct ParameterInfo
{
    ParameterInfo (const String &parameterID, 
                   Category parameterCategory=AudioProcessorParameter::genericParameter);
}

AudioParameterFloat (ParameterInfo (paramID)
    .withLabel ("Gain")
    .continuous (true)
    .metaParameter (false), 
    range, defaultValue);

if the range and default value should be part of the struct is up for debate, they don’t apply to all parameters.

[reuk]

Thanks @daniel and @yfede, these are both very good suggestions. I think I can probably roll them both into the change.

[yfede]

Designated Initializers intensifies

[jimc]

Hey @reuk - is this likely to make it into the develop branch soon. We are about to do some changes that require new paramters and I’d be keen to avoid a local JUCE branch if possible.

Thanks!

[reuk]

It should be out shortly, yes. I was hoping it would land this week, but we’ve got a bit of a backlog of work to review at the moment so it might end up being next week instead.

[reuk]

Thanks for your patience with this issue. We decided that it would be best to push a point release before landing this change, as this will require fairly significant changes for anyone distributing AU plugins for use in Logic.

The relevant changes are listed below:

AudioProcessorParameter: Add getVersionHint function

AU Client: Order parameters according to version hint

AudioProcessor: Validate parameter version hints

AudioProcessorParameter: Add new ParameterID and Attributes types

As always, please let me know if you find any problems in these changes and I’ll do my best to fix any issues.

[aguy-PG]

Thank you @reuk!

It looks like there is an assertion error when I try to create a plugin instance (code below was working before this update):

AudioPluginFormatManager formatManager;
formatManager.addDefaultFormats();
String errorString;
plugin = formatManager.createPluginInstance(desc, 44100.0, 512, errorString);
return errorString;

It looks like juce_AudioUnitPluginFormat.mm calls setHostedParameterTree() (line: 1563) but the code was not updated to pass a version hint so the assertion is hit. Is there a way for me to pass a version hint when I create a plugin instance in my program?