Issues with version integer reported by VST2


#1

The code that returns the version number for the VST2 format is the following (found in the VST wrapper):

  #if JUCE_VST_RETURN_HEX_VERSION_NUMBER_DIRECTLY
    return (int32) hexVersion;
   #else
    return (int32) (((hexVersion >> 24) & 0xff) * 1000
                     + ((hexVersion >> 16) & 0xff) * 100
                     + ((hexVersion >> 8)  & 0xff) * 10
                     + (hexVersion & 0xff));
   #endif

where hexVersion is the JucePlugin_VersionCode, which is usually a hex number in the format 0xMMmmBB.

For example, for version “1.2.3” the matching JucePlugin_VersionCode is 0x010203.

In the code above, the JUCE_VST_RETURN_HEX_VERSION_NUMBER_DIRECTLY macro has been added in January 2013 but there is no documentation about how or when that should be set.

I have old projects that are now built with JUCE 4 which don’t have it defined.
Also a brand new project generated with the latest Projucer (which I don’t normally use) does not #define it, I have checked just to be sure.

Now, when that macro is not defined (most of the time, as per default), the above code for the version number is prone to a subtle bug: when one of the components of the version number reaches 10, the returned value is “wrong” meaning that it represents a different version that’s not the one intended.

For example, if I specify version to be “1.2.11” in Projucer, the generated JucePlugin_VersionCode is correctly valued 0x01020b, but the code that is executed when JUCE_VST_RETURN_HEX_VERSION_NUMBER_DIRECTLY is not set emits a value that’s equivalent to the value that would be emitted if the version was “1.3.1”.

This is confirmed by the following screenshot, taken with Cubase 9 for a VST plug-in generated with the latest Projucer:

This causes more issues than just wrong version number appearing: there are DAWs (Studio One for sure) which won’t load the plug-ins saved chunk upon session load if it detects that the current reported version of the plug-in is lower than the one reported by the plug-in when the chunk was saved.

This actually happened to me: I had my plug-in version set to 1.2.11, and when I released version 1.3.0 it refused to load the settings of sessions created with 1.2.11 because Studio One thought that the one previously used was 1.3.1 instead of 1.2.11.

Setting JUCE_VST_RETURN_HEX_VERSION_NUMBER_DIRECTLY seems to resolve the issue, but I think that:

  1. an assertion is required to ensure that each component of the version number is less than 10 when JUCE_VST_RETURN_HEX_VERSION_NUMBER_DIRECTLY is not set

  2. proper documentation should be provided about what JUCE_VST_RETURN_HEX_VERSION_NUMBER_DIRECTLY does and when it is suggested to set it.

Besides, in Cubase setting it gets rid of the leading “0.” that’s displayed in the version number. I think that Cubase does some guesswork to understand whether the returned code is in “base 10” or “base 16”.

More on that subject and about people hitting bugs similar to mine are visible if you search the forum for “JUCE_VST_RETURN_HEX_VERSION_NUMBER_DIRECTLY” or JucePlugin_VersionCode and VST2


Proper detection of Cubase version
Plugin state not restored in Cubase 8 for newer plugin version?
#2

See also this discussion in which a solution was proposed for separating the chunk version from the plug-in version via JucePlugin_VSTChunkStructureVersion:
https://forum.juce.com/t/juce-projucer-and-vst-version-vendor-version/?source_topic_id=23867

The workaround was integrated into the develop branch, but without explicit ProJucer support because “we want to add more robust version information to audio processors in future versoin of JUCE”


#3

Thank you @frankfilipanits for mentioning the JucePlugin_VSTChunkStructureVersion thing, I encountered it while looking for the places where the version is passed to the host in VST2, but didn’t take the time to investigate what that did.

I think that’s another macro that would have benefitted from a short accompanying comment to explain its purpose.

However, I think that the core of the problem still remains also if I were using JucePlugin_VSTChunkStructureVersion because of the ambiguity of how that version code may be interpreted anyway: if I specify 0x01020b for the chunk structure version, meaning that it is revision 1.2.11 of the chunk format, but Cubase interprets it as 1.3.1, then when I upgrade the chunk format version to 1.3.0, Cubase will still refuse to load the chunks created with chunk format version 1.2.11 because it thinks they are newer.
What saves the day in the case of the chunk version is that you may never hit such strange version numbers, probably limiting to Major.minor.0 version numbers.

My intention for today is to reverse engineer the guesswork performed by Cubase to decode the version number from the integer that’s provided by the plug-in, and maybe check if other DAWs follow the same convention too.
Unfortunately, the VST2 documentation does not provide an official encoding for that integer, and I think that’s the real root of the problem here.


#4

Just to check that I fully understand the problem:

I don’t quite see how JUCE_VST_RETURN_HEX_VERSION_NUMBER_DIRECTLY resolves the problem. Don’t a lot of hosts expect the version to be in “base 10” (who on earth thought it would be a good idea to have the version number be in “base 10”?!?)? Using JUCE_VST_RETURN_HEX_VERSION_NUMBER_DIRECTLY would really just fix it for Cubase, right?

I absolutely agree we need to add a few asserts. But, in summary, it means that you just can’t have a major/minor/build version number that is 10 or larger?


#5

I’m working on it, I will have a definite answer for your questions in few hours tops!


#6

Ok, I’ve worked this out.

Let’s call VERSION_CODE the number that is returned by the VST2 plug-in to inform about its version.

That is the number expected to be returned by the AudioEffectX::getVendorVersion() method, which is handled/implemented by handleGetManufacturerVersion() in the JUCE VST2 wrapper.

Ok, let’s state my simpler finding first: both Cubase and Studio One 3 don’t feed the chunk stored in the session if the plug-in VERSION_CODE is lower than the one emitted by the same plug-in when the session was saved.
If the VERSION_CODE is equal or higher, it is passed to the plug-in the usual way.
A simple numerical comparison takes place in this case, and VERSION_CODE is treated as a simple integer.
In this context, no interpretation of which part represents Major, Minor and Bugfix version number is performed: the comparison is plain arithmetic.

This may seem obvious for now, but that’s not really that obvious when you see how “strange” the following is.

Of all the hosts I have tested, Cubase is the only one which also attempts to display a version number for the plug-in, inferring it from that plain VERSION_CODE integer, and that’s where madness begins.

Probably because the VST2 does not specify a format for encoding Major, Minor, Bugfix (and even Build) in that VERSION_CODE, I guess various developers followed different conventions and Cubase tries its best to make sense of them all.

What I found by attempting with various VERSION_CODE values is the following:

  • Encoding A: when in the range [0, 9], the value is interpreted as a Major version number. All the following dot values are 0. value 1 corresponds to “1.0.0.0”, 2 to “2.0.0.0” and so on.

  • Encoding B: when in the range [10, 9999], each decimal digit of the value is treated as a dot component, meaning that the number 1234 is interpreted as “1.2.3.4”

  • Encoding C: when in the range [10000, 65535 (0xFFFF)], the four most significant decimal digits are treated as the dot components. The digit for the units, which is the fifth, is ignored.
    12345 is shown as “1.2.3.4”

  • Encoding D: when in the range [0x10000 (65536), 0x64FFFF], the three bytes that compose the hexadecimal representation of the number are treated as the three most significant dot parts, leaving the fourth always at 0.
    0x010203 corresponds to 1.2.3.0.
    Note that 0x64 is 100, so the highest major version number for this representation is 100.

  • Encoding E: when in the range [0x650000, MAX_INT] the digits of the decimal representation of the number are treated as follows: MMMmmBBbbb, where MMM, mm, BB, bbb are the four dot components.

(I made up the letters for the different encodings, it’s not like they are documented anywhere at all)

So, in the end what should be done?

My personal suggestion is this: because the JucePlugin_VersionCode is natively in Encoding D, that is the encoding that should be used (whenever possible) to report the value to be displayed to Cubase.

I added “whenever possible” because “Encoding D” cannot be effectively used to represent all version numbers, in particular version numbers that start with “0.”

For example, expressing “0.1.0” in Encoding D yields 0x000100, which is decimal 256.
The problem is that the value 256 falls in the range of values that are interpreted according to Encoding B, giving “0.2.5.6”.

So, my complete proposal is implemented as follows:

static inline int32 convertHexVersionToDecimal (const unsigned int hexVersion)
{

   #if JUCE_VST_RETURN_HEX_VERSION_NUMBER_DIRECTLY
    return (int32) hexVersion;
   #else

    const int32 major = (hexVersion >> 16) & 0xff;
    const int32 minor = (hexVersion >> 8) & 0xff;
    const int32 bugfix = hexVersion & 0xff;

    if (major < 1)  // in this case attempt to use Encoding B, with assertions for values out of range
    {
        jassert (minor <= 9);
        jassert (bugfix <= 9);
        return major * 1000 + minor * 100 + bugfix * 10;
    }

    if (major > 100)    // in this case attempt to use Encoding E, with assertions for values out of range
    {
        jassert (minor <= 99);
        jassert (bugfix <= 99);
        return major * 10000000 + minor * 100000 + bugfix * 1000;
    }

    return hexVersion;  // in all other cases, Encoding D well represents the value.
   #endif
}

(Disclaimer: I haven’t compiled this code yet, it’s to be treated as pseudo-code)

I have left in there the JUCE_VST_RETURN_HEX_VERSION_NUMBER_DIRECTLY to allow developers that already have resolved this in their own ways to override the default behavior, and the default behavior is such that is still a monotonic function whose outcome increases when the JucePlugin_VersionCode increases.

For all cases where the native encoding of JucePlugin_VersionCode cannot be used, assertions are in place to ensure that the dot parts of the version number are within the desired limits.


#7

Wow, this post is gold! Thank you!!!

It might be worth posting the table on non-JUCE forums as well. I think any VST2 developer would be interested in this! I can’t believe this is not documented anywhere.


#8

I posted it already on KVR because I stumbled on one topic about this while unsuccessfully searching for an answer.

Just for reference, it’s here: https://www.kvraudio.com/forum/viewtopic.php?f=33&t=359599&start=15


#9

And to answer this questions of yours: the problem usually encountered when JUCE_VST_RETURN_HEX_VERSION_NUMBER_DIRECTLY was not set was that, when the Minor or Bugfix part exceeded 9, a code is generated that represents a version higher than expected.

So for example, 1.2.11 (JucePlugin_VersionCode = 0x01020b) is transformed to 131.
When updating to 1.3.0 and JUCE_VST_RETURN_HEX_VERSION_NUMBER_DIRECTLY still not set, this yields the value 130 and causes the DAW to not feed the plug-in the chunks created with ver. 131.

If at that point you define JUCE_VST_RETURN_HEX_VERSION_NUMBER_DIRECTLY, your version 1.3.0 now directly reports the integer 0x01020b as its version to the host, which is various orders of magnitude bigger than the mistakenly generated 131, thus making the newer version of the plug-in receive the chunks created with version 131 as well.

But the workaround of defining JUCE_VST_RETURN_HEX_VERSION_NUMBER_DIRECTLY to resolve this case would not be necessary any longer with my code above.


#10

Thank you for your code. I’ll add it to JUCE. I really like that it falls back to what JUCE had before, if the version code can be represented by Encoding B. If not the number returned will always be larger than what people had before. This will ensure that chunks are still loaded.

However, I think there is an issue with your code. Shouldn’t the first if statement read:

if (major < 10)  // in this case attempt to use Encoding B, with assertions for values out of range

#11

Please for the love of all that is holy, hold off a second before adding this to JUCE. This is a complex issue and while it’s great that yfede has done this additional investigation, the opportunities for royally screwing things up for other hosts are abundant.

The ROLI statement from last year that JUCE could use more robust version information is correct - it is a sticky issue that deserves to be addressed thoroughly and thoughtfully.

Specifically, this particular fix needs to exhaustively tested against many hosts, and the CHUNK version still needs to be decoupled from the PLUGIN version. This is part of the “original sin” of JUCE’s VST2 version handling. The CHUNK version should only be incremented when the CHUNK format changes incompatibly, NOT with every plug-in rev, and it should be a plain integer, not a dotted trio or quad of any sort.

There is also the broader issue of versioning, since OSX really wants to keep it limited to dotted trios (sometimes with an separate ever-increasing build number), but Windows likes dotted quads. And if you want to get fancy, it would be nice to support Semantic Versioning at the highest level, with that mapping down to dotted-trio, dotted-quad, and VST2 random-base formats as appropriate.

Please, please, please don’t just shove another patch in on top of the existing patches.


#12

100% agree with this. Please keep it separate from the plugin-version and ideally adjustable through the Projucer.

In NEXUS case we prefer to keep the chunk-version between even version 1 & 2 the same and then let the plugin handle any up/down-grades and conflicts. Much safer that way.


#13

No, it was correct with the 1:

Version numbers with major = 1 are already well displayed using Encoding D, which well represents version numbers ranging from 1.0.0(.0) to 100.255.255(.0)

Where Encoding D falls short is in representing version numbers with major = 0 (thus < 1), which may be typical in beta/preliminary versions, or open source

Take 0.1.2 for example. If you encoded it with Encoding D, you would get 0x000102, which is < 0x10000 and hence falls out of the range of values decoded using Encoding D. In fact, 0x102 is nothing but decimal 258, and if you feed that to Cubase, it would interpret it according Encoding B, not D, hence displaying version “0.2.5.8”.

That would result in a wrong display (remember that we wanted to display “0.1.2”), thus only for the version numbers whose major is 0, we should resort to Encoding B and thus obtain the integer to emit as follows (with the limiation that, in this case, minor and bugfix must not exceed 9):

major * 1000 + minor * 100 + bugfix * 10.

It may be objected that in my code above major would thus always be 0 in that case, and thus it may omitted from the calculation above. I left it there for readability and clarity reasons.

I also side with the others commenting here, that this changes in this area should be well discussed and perhaps tested on a separate experimental branch, like the multibus stuff was, because they may have unforeseen implications and is thus better to get them right before merging it in even in develop.


#14

One more thought that is in a way collateral to the separation between plug-in version and chunk format version:

Unless I’m disproved by additional evidence, the “Encoding” algorithm above is only used for displaying the version number (and, even more, Cubase is the only DAW that I found which displays a version number, so we could even consider restricting those conversions only if the detected host is Cubase).

My point is that, in the context of versioning of the chunk, whether we use JucePlugin_VersionCode or a separate value, the version code/value does not need this “mangling” at all!

Which means that IMHO convertHexVersionToDecimal() could even be avoided in the following code:

   #ifdef JucePlugin_VSTChunkStructureVersion
    vstEffect.plugInVersion = convertHexVersionToDecimal (JucePlugin_VSTChunkStructureVersion);
   #else
    vstEffect.plugInVersion = convertHexVersionToDecimal (JucePlugin_VersionCode);
   #endif

and only left in the code that responds to the callback mentioned in my previous post, which corresponds to the getVendorVersion().


#15

Just a follow up: I’m working on and testing a patch for this, hopefully I will be able to submit it here for consideration in the next few days


#16

I’m sorry for the length of the post ahead. Unfortunately the subject is complex and I wanted to make it as clear as possible.

So, I have finally done some testing and reached some conclusions.

My suggested patch is provided by this pull request:

Essentially, it revolves around three distinct changes.
I’ll list them here, and below the list explain why I think they will not break any existing session saved with plug-ins compiled with the previous code.

  1. Initialize the vstEffect.pluginVersion in the constructor with the value of JucePlugin_VersionCode (or JucePlugin_VSTChunkStructureVersion if that one is used) like this:

    #ifdef JucePlugin_VSTChunkStructureVersion
     vstEffect.plugInVersion = JucePlugin_VSTChunkStructureVersion;
    #else
     vstEffect.plugInVersion = JucePlugin_VersionCode;
    #endif
    

    without any “mangling” by the “conversion funcion” which was used before.

  2. On the other hand, the value returned by handleGetManufacturerVersion() should still be passed through the conversion function, like this:

    pointer_sized_int handleGetManufacturerVersion (VstOpCodeArguments)
    {
        return mangleHexVersionForDisplay (JucePlugin_VersionCode);
    }
    

    I have renamed such function mangleHexVersionForDisplay() for reasons given below.

  3. The body of that convertion function is essentially the following (stripping comments):

    static inline int32 mangleHexVersionForDisplay (const unsigned int hexVersion)
    {
       #if JUCE_VST_RETURN_HEX_VERSION_NUMBER_DIRECTLY
        return (int32) hexVersion;
       #else
    
        const PluginHostType hostType;
    
        if (hostType.isSteinberg ())
        {
            const int32 major = (hexVersion >> 16) & 0xff;
            const int32 minor = (hexVersion >> 8) & 0xff;
            const int32 bugfix = hexVersion & 0xff;
    
            if (major < 1)
                return major * 1000 + minor * 100 + bugfix * 10;
    
            if (major > 100)
                return major * 10000000 + minor * 100000 + bugfix * 1000;
    
            return hexVersion;
        }
    
        return hexVersion;
       #endif // JUCE_VST_RETURN_HEX_VERSION_NUMBER_DIRECTLY
    }
    

    which basically means: just return the hexVersion intact, unless we are in a Steinberg DAW: in that special case, do some manipulation on it in some corner cases (major < 1 or > 100) in order for the version number to display correctly, because Steinberg DAWs need that, as shown in my post above.

Now I’m going to answer some possible doubts and concerns about the proposed change, but first, let’s take a moment to consider that this change only affects the VST2 wrapper code.
DAWs that only use other formats (Logic, Pro Tools) will not be affected by this change at all. And that’s good, because they already have their idiosincracies.


Q: This change implies that the value returned by handleGetManufacturerVersion() could differ from what is set in vstEffect.plugInVersion upon initialization. Can this cause problems?

A: No, that is not expected to cause problems, because:

  • This situation can already happen even with the current code, when one uses a JucePlugin_VSTChunkStructureVersion that’s different from JucePlugin_VersionCode, and doesn’t seem to have caused any problem so far

  • If one uses only JucePlugin_VersionCode, then the two values will only differ inside Steinberg DAWs. I have done some tests with them, and the plug-ins never showed any malfunction.


Q: My plug-in was compiled with JUCE_VST_RETURN_HEX_VERSION_NUMBER_DIRECTLY = 1, will something change for me?

A: no, nothing will change for you: the behavior of that macro is retained so that if you define it to 1, you are guaranteed that your hex code is passed intact rather than manipulated. It’s just that after this change, there are less places where the manipulation could occour.


Q: My plug-in was compiled without setting JUCE_VST_RETURN_HEX_VERSION_NUMBER_DIRECTLY, what will change for me?

A: Everything will work as expected, and this is why I believe so:

Now, let’s consider the behavior of the convertHexVersionToDecimal () function as it is now:

static inline int32 convertHexVersionToDecimal (const unsigned int hexVersion)
{
    return (int32) (((hexVersion >> 24) & 0xff) * 1000
                     + ((hexVersion >> 16) & 0xff) * 100
                     + ((hexVersion >> 8)  & 0xff) * 10
                     + (hexVersion & 0xff));
}

It is easily seen that this function returns a value that is:

  • equal to hexVersion, when hexVersion is between 0 and 255
  • less than hexVersion, when hexVersion is 256 or above.

This means that initializing vstEffect.plugInVersion directly with the version hex code, rather than passing it through the conversion funcion (change #1), will always result in a vstEffect.plugInVersion equal or greater than the one obtained with the previous code.

According to my tests (table below), an increment in the value of vstEffect.plugInVersion will never prevent the plug-in from receiving the data saved in the session.

I have tested it only with the mentioned DAWs because that’s what I could have easy access to.

As you can see, only Studio One and Cubase seem to care about that value, and they refuse to pass the data only when plug-ins report a lower value. When the value is equal or higher, everything works as normal.
The other two major DAWs that I have tested with, Ableton Live and Cakewalk SONAR, simply ignore that value, so there’s no problem for those either.

Please also note that the comparison performed is purely arithmetic: if Cubase took into account the version number encoded by the version code, the tests 6 and 7 would have had the opposite outcome.


These tests also show that what’s returned by handleGetManufacturerVersion() is not taken into account in any of those hosts.
Studio One 3 does not even call that function, and Cubase only does so during the scanning of the plug-ins, probably to obtain the version number to be displayed, according to the rules of previous posts.

In consideration of this, letting handleGetManufacturerVersion() return the version code unmodified in most cases, with the sole exception of Steinberg hosts (behavior implemented in mangleHexVersionForDisplay()) seems quite safe to me.

Additional special cases for other DAWs could be added to that funcion if that proves necessary, and I believe that putting this change on a separate branch for experimentation and testing is the safest and most desirable choice, also to bring those special cases to the surface if they exist.

EDIT: I have intentionally left out of this post the discussions about:

  • whether it’s a good idea to move the entire body of mangleHexVersionForDisplay() inside handleGetManufacturerVersion(), because that’s the only remaining sport where it is called from
  • how to warn the developer if the JucePlugin_VersionCode that he is using is breaking some of the constraint that would lead Cubase to misinterpret the version number to display.

I’ll leave these (especially the second one) for later.


#17

if (hostType.isSteinberg () == false) doesn’t match unless we are in a Steinberg DAW, unless I am missing something.


#18

yes sorry, that was a bad copy and paste from an earlier formulation which used that condition for an early return.

the == false should be removed, or toggled to true. I’ve edited it now in my post above.
The code in the pull request was correct already.


#19

Gentle bump.
Is someone in the JUCE team pondering about my PR described above?


#20

Hi @yfede,

Sorry for losing the ball on this and once again thank you for the incredible amount of work you’ve put into this.

I’m looking at your PR now. I’m a tad bit worried about removing the convertHexVersionToDecimal here:

   #ifdef JucePlugin_VSTChunkStructureVersion
    vstEffect.plugInVersion = convertHexVersionToDecimal (JucePlugin_VSTChunkStructureVersion);
   #else
    vstEffect.plugInVersion = convertHexVersionToDecimal (JucePlugin_VersionCode);
   #endif

As you write above, this may cause vstEffect.plugInVersion to increase in value after applying your PR for some plug-ins. I understand, that - according to your tests - this will not prevent plug-ins loading saved data. However - although I understand that handleGetManufacturerVersion should be used to display the version to the user -, I could imagine that some hosts will actually show vstEffect.pluginVersion.

This really wouldn’t be very good if hosts all of a sudden display a different version than the user expected to be installed.

Can you comment on in this potential problem?