When the JUCE team introduces breaking bugs in plug-ins

[fr810]

Every now and then, the JUCE team (well mostly me really ) introduces a bug into the JUCE code which can only be fixed by a version breaking change to the plug-in code.

By “version breaking”, I mean that there will be some sort of incompatibility with old versions of your plug-in - for example, a Cubase project saved with an old version of your plug-in may fail to load when using a newer version of your plug-in compiled with a newer version of JUCE. Examples of such “version breaking” fixes are here and here.

To avoid only talking hypotheticals, let’s take the former example: we noticed that some hosts don’t like it when the parameter ids are negative. In fact, the VST3 SDK strongly suggests that parameter ids should only be positive. Let me be clear: allowing negative parameter ids was an oversight and a really unfortunate bug introduced by the JUCE team.

But how to move forward in such a case? The only workaround is to avoid negative parameter ids in JUCE which can easily be fixed. The downside is that the fix will “version break” your plug-in.

Therefore, JUCE’s current policy is to apply the fix by default - but add some sort of option to opt-out of the fix.

The reasoning behind this choice is the following:

1. If there is a bug in JUCE that can only be fixed by “version breaking” we need to push this out ASAP. Otherwise, more and more plug-ins will come into the market that are not only potentially broken - but hard to fix without “version breaking”. Therefore, the default must be that the fix is applied.
2. Some people have suggested to always ask the user if the bug-fix should be applied or not. But this is overly confusing for a vast amount of JUCE users. Many users might not even understand the technical details of the bug - nor do they need to. As the JUCE team introduces more and more “version breaking” changes (this is bound to happen), the number of “questions” the Projucer would need to ask will become overwhelming. In addition, the JUCE users that are likely to run into any “version breaking” problems have already released products to the public and therefore are usually more experienced JUCE users and have mostly interacted in the forums before. It’s easier for these developers to find the fix to a specific “version breaking” problem.

What are your thoughts on this? Any suggestions on how we could improve when we fix these types of bugs?

[yfede]

In this post you are somehow implicitly referring to usage of Projucer.
The fact is, that some of us don’t use it at all, or use it only to bootstrap projects. In consideration of that, I’d say that Projucer would be an insufficient advertiser of those changes anyway.

In my opinion, it is always better to introduce breaking changes in a way that requires developer action when the offending commit is merged in/cherry picked, and surround the corresponding area of code with a clear comment about what has happened.

Also, such breaking changes should be properly advertised here on the forum too (which has been done pretty well in the past) and a link to the corresponding topic should be added to the existing topic that lists breaking changes (I’m going to search it and edit this post with the link later).

EDIT: here is that topic (didn’t remember it was this old and this ‘stale’): List of changes that impact user-code (since 1.52 release)

[pflugshaupt]

I find these changes very frustrating. The only way to be sure nothing funny happens is to follow the changelog of JUCE very closely or not update at all. If you just follow the master branch there’s a high change such changes are completely missed. To be sure you also have to closely follow the forum and frankly it’s all a bit too time consuming in my opinion.

There would be ways you could prevent silent breaking in code. This opt-out thing is not a good idea imho. Instead you should make things break on purpose for old code once compatibility breaks. Of course newly created projects should not ever break or ask the user questions. But you have it in your hands to change the newly created projects accordingly. A simple scheme could just use some preprocessor macros to know which JUCE version the project was created with. And maybe the user could adjust a preprocessor define to control the level of warnings. Similar to how windows and osx feature level defines work.

[pflugshaupt]

@fabian

If there is a bug in JUCE that can only be fixed by “version breaking” we need to push this out ASAP. Otherwise, more and more plug-ins will come into the market that are not only potentially broken - but hard to fix without “version breaking”. Therefore, the default must be that the fix is applied.

This IMHO not logical at all. These changes also happen to break all the JUCE plugins in existence in case the developers need or want to update JUCE and build a new version. Which is a lot more than new plugins coming into the market.
At the same time if updates tend to break existing plugins, developers will be less likely to update JUCE and therefore the number of broken plugins on the market is unnecessarily increased.

I think it’s in everybodies interest to have as many plugins use as new a version of JUCE with as little bugs as possible, but we developers need to be able to trust updateing won’t break compatibility without warnings.

[bld]

IMO if you have to do some “breaking changes” you should always:

1. Provide some way of backward compatibility (like you did with AAX pluginIDs enumerations)
2. Warn clearly about it in the ChangeList.txt. It would be also good to update it with such warnings even on develop branch. With new tagged version you could move them to the changes but anyone who follows develop often would be warned without reading all the commit messages.

[chkn]

Here is my Proposal:

When you save a Projucer project, the current “API-Level” of used JUCE should be saved into the project. (Maybe in the Appconfig.h, or *.jucer file (not the jucer version))

When a breaking change is committed, the API-Level should be increased immediately.

If you upgrade your juce-source, and projucer checks, that the same project was saved with an older api-level, and the new juce has breaking behavior changes (relative to the old api-level), projucer should add something to the code, which breaks the compilation, with a hint.

BTW: It would be cool if all the macros would be replaced bei const-expressions, which are bundled in a global config class

[lalala]

I don’t want to go through 5 forum threads everytime I open the projucer and update juce/a plugin.
Ideally, I’d like a short document that sums up the settings that I should turn on or off to avoid breaking, and what that implies.

Something like that for instance (nb : what follows may be totally wrong, it’s just an example)

In order to not have version-breaking :

• If your plugins used audio parameters not inheriting AudioParameterWIthID, then enable JUCE_FORCE_LEGACY_PARAMS

  note however that plugins with JUCE_FORCE_LEGACY_PARAMS enabled have the folowing bugs :
  - if you add a param, you may break the automation
  - and another terrible bug description

• If JUCE_FORCE_LEGACY_PARAMS is not enabled and your plugin was first made with Juce < 4.3, then disable JUCE_USE_STUDIO_ONE

note that plugins with JUCE_USE_STUDIO_ONE disabled have the folowing bugs :
- blabla

• If whatever, do that

  note that you will then have the following bug : 
         - blabla

also, I’d like the default settings to be specified in the projucer. For now it’s written “(USE DEFAULT)”. that would be cool if that could read “(USE DEFAULT - i.e. ENABLED)”

[KoenTanghe]

I like Christian (chkn)'s ideas above.

For me, it’s important:

• to have 1 central place where we can quickly find out which breaking changes have occurred, mapped on a timeline with JUCE API version numbers (without having to skim through a whole git log)
• to have a description of these breaking changes: what was the behavior before and what is it now after the change (not only the technicalities, but also the implications from an end-user perspective)
• have a way to know which JUCE API version is being used from within your code (that might already exist?) so we can base decisions on that ourselves as well
• always increment the JUCE API version after such a breaking change (indeed also on the development branch)!
• if there is a way in the JUCE code or the Projucer to know that you’re updating to a newer JUCE version that has breaking changes from the last version a plugin used, then by all means show us a warning when building the plugin (or perhaps indeed an error as some people suggested)

With this, I think we should be able to deal with these kinds of situations (don’t we all end up there with our own code at some point as well… That’s why version numbers are important…)

Of course, the number one thing remains: please avoid breaking changes if there can be workarounds made in JUCE itself (but I understand we’re talking about cases where this is really not possible).

[fr810]

I’ve now added a BREAKING-CHANGES.txt document to the develop branch of JUCE. This will eventually go on the master branch as well. Any feedback is welcome!

[KoenTanghe]

Thanks @fabian. This should help!

One question to clarify something about the oldest change in the breaking changes list (and if you feel this belongs in another place on the forum, feel free to move this):

If the last release of the plugin was done under these conditions:

• JUCE version before 4.2.1
• plugin did not make use of AudioProcessorParameter(WithID) (only using plain old setParameter/getParameter) [clarified in post edit]
• only a VST 2.4 version was released (no VST 3)

then, will upgrading JUCE to the last development version and building/releasing a new VST 2.4 plugin still cause the described problems related to that bypass parameter?
And is that bypass parameter the only one, or are there others and other effects to consider/look out for?

Thanks,
Koen

[ttg]

Would be really helpful to make sure you organize proper changes between releases.[quote=“fabian, post:9, topic:20941”]
BREAKING-CHANGES.txt

JUCE now uses the paramID property used in AudioProcessorParameterWithID to uniquely identify parameters to the host
[/quote]

From what I recall (and see) this is related only to newer methods of plug-in management. but for plain addParameter it doesn’t require any change.

And about best workflow, as @KoenT suggest.
having a definition of JUCE_API with numeric value sounds to me the best resolution.
So…

1. JUCE_API will be generated on new Projucer project but won’t change without explicitly fiddling with it.

2. You (aka JUCE team), can add compiler warning when the API is older in a way you discourage the use of it or even fail to compile if feature has dependency on newer changes.

That way -

• upgrading JUCE version should be a hassle.
• I can keep pace on changes easily.

[chkn]

@ttg

From what I recall (and see) this is related only to newer methods of plug-in management. but for plain addParameter it doesn’t require any change.

No (imho) Breaking Change: Audio Parameter ID Support - #14 by chkn

[ttg]

Bahhh…
Just investigated this so…

• it BREAKS VST/AU for many hosts.
• Logic X strangely manage to cope with this? didn’t investigate further…
• you can use auval to see the ids returned of your plug-in before/after (seems interesting to add this to tests on build systems…)

[yfede]

@ttg

• you can use auval to see the ids returned of your plug-in before/after (seems interesting to add this to tests on build systems…)

You mean the ids of the parameters? I am interested in this, can you elaborate?

[ttg]

auval -v TYPE name manu | grep -E 'Parameter ID:'

(you can do this without the grep…)

So you get details of your params.

Parameter ID:48
Parameter ID:49
Parameter ID:50
Parameter ID:51
Parameter ID:52
Parameter ID:53

[KoenTanghe]

In the case I asked about above, the plugin just uses getParameter/setParameter (it doesn’t use addParameter/AudioProcessorParameter(WithID) at all). I’ve edited my post to clarify that.
So, would there be a problem when upgrading such a VST 2.4 plugin to the latest development branch then?
It’s just not entirely clear to me if that bypass parameter issue affects plugins that handle their parameters themselves (I already had my own Parameter classes).

[patrickkunz]

I’m using the new parameter system and it looks like all id’s are positive:

Parameter ID:11288566 Parameter ID:28943097 Parameter ID:69879911 Parameter ID:70051149 Parameter ID:70103816 Parameter ID:95852938 Parameter ID:110364447 Parameter ID:110364453 Parameter ID:260525163 Parameter ID:351093686 Parameter ID:421446943 Parameter ID:504281783 Parameter ID:505906535 Parameter ID:505906541 Parameter ID:512381614 Parameter ID:665490880 Parameter ID:824404411 Parameter ID:1232173611 Parameter ID:1336600346 Parameter ID:1650207775 Parameter ID:1676421509 Parameter ID:2068465703

Do i still need to switch on the compatibility mode when i’m updating juce or stay the positive id’s the same?

[ttg]

Ok,
Some things to remember:

• auval will prefer your home folder components! so make true sanity against released builds by testing returned IDs of older builds.

• @KoenT , VST2 wrapper doesn’t use the JUCE_FORCE_USE_LEGACY_PARAM_IDS flag.
  @patrickkunz and others, the main concern is for VST3 and AU. both started using hashes IDs.
  so instead of getting 0…N for parameter IDs you’ll get some numeric values based on the hash function used in the wrapper.
  I saw that for AAX (and RTAS?) it should do much but for AU & VST3 if you released prior to the changes AUTOMATION WILL BREAK (with the exception of Logic X from my testings).

[lcapozzi]

semi OT question… but what’s the safest JUCE version today? I’m still on 3.2 and I’m very afraid to step to a newer version. I would like to use the multibus, but so far I don’t know if that’s still experimental or there’s a working version… and, if exists, the related JUCE version what will break for my existing plugins.

[yfede]

Are you managing your projects with Projucer or not?