There’s a bug in VSTPluginInstance::prepareToPlay() that hinders some plugins from reporting correct latency after calling prepareToPlay(), most notably the UAD plugins.
You need to put this line:
at the very end of the prepareToPlay() function, not at the beginning.