Is there a generally accepted maximum data size and return time for the data provided to the host in ‘getStateInformation’?
I’ve had a (Windows) user complain of brief UI pauses (e.g. a fraction of a second) whilst Bitwig does background requests for the plugin state (I observed it does it once per minute in Bitwig 4.4.2) whilst the plugin UI is open (and have seen this on rare occasion myself, but only in Bitwig/Windows).
My personal feeling on this is that the getStateInformation should be called by the DAW at a low enough priority (if being done as background process rather than when saving the entire state of the DAW project) that it would never hold up UI drawing - but it seems that might not be the case in Bitwig.
I can of course take some steps to reduce the size of data returned (in some cases my state information can include a small amount of audio sample data), although this issue has not affected any customers other than the one person that is experiencing it.
Do you implement
getCurrentProgramStateInformation()? It’s possible that implementing this could make it a bit cheaper for the host to periodically save the current state…
We have one product that allows embedding small samples inside the state. I didn’t add this feature to our newer product anymore. I think most hosts are not designed to handle bigger states.
There are a few problems I noticed:
- Most time all data is saved within one session file. Loading and saving of the session gets slow. You have a huge session file of a few MByte pretty fast.
- The chance to get a corrupt session file is bigger.
- Hosts request the state all the time in the background. This uses resources also when done with low priority or in another thread. It’s not possible to split up the work. The hosts always request the whole state.
- You never know how many instances of your product the user loads. Things sum up.
- You probably do not know how big the state is (how big the samples are for example).
CLAP probably has a solution for this in the future, where the DAW asks the plugin for resources and saves them near the project file.
That sounds awesome …
I think SampleData should never be embedded into the host. This is an issue many plugins have to solve and if not every single one would store the samples into the host, the user would still need to figure out a solution depending on their work station setup.
So IMO you should just support what other plugins are doing to streamline the users workflow and call it a day.
I didn’t implement it, but in my case I think it would be the same amount of data being returned.
I think I’m going to have to disable or at least provide the option to turn off small sample embedding, however even without sample embedding the user I mentioned reported seeing regular brief UI pauses (mouse busy indicator briefly flashes up, at the observed frequency of the requests for the state - every 60s) even with a very simple plugin state (where the serialized data block which describes the patch is around 2K bytes).
This user is running Bitwig 4.4.10 and I wonder if maybe there is a bug there with file operations holding up UI draws due to the state retrieval. I will do some profiling on this background request anyway - but I don’t understand why it would not be at a lower priority than UI drawing.
In the case of this particular product, the patches are modular and there can be between 1 and 16 layers of modular patches - so depending on the number of nodes and layers, the state description can vary greatly in size - even without samples a state descriptor can be 10s of Kilobytes - state descriptors can also include user text (for example there is a node that can show user entered text notes).
This image shows my plugin running in Bitwig (real-time logging to buffer and dumped to file on plugin close - this is a custom trace that I can view in Perfetto).
We can see that even though my plugin is set to run in single threaded mode, there are multiple threads triggering calls to processAudioBlock (the blue lines) - I see the same behavior in Ableton 11, but Reaper only processes the plugin audio on one thread (at least in Windows 11 as of my testing today).
What I am a bit surprised about is that the background/regular request for getStateInformation from Bitwig appears to run on the same thread as the UI/Message thread of the plugin (the thread with all the green lines includes logs I added for timer callbacks and things like createEditor/resized - but also seen on the same thread is getStateInformation at regular intervals).
A bit of additional information, running the same test build in Ableton 11, I can see that even after 5 minutes there were no calls to getStateInformation - it only gets called on plugin close.
Further note - I think I found a bug in my code that is triggering the mouse busy cursor during the save state request from the host - and there are some optimizations I will make to reduce the time it takes.
Hopefully that will be enough to prevent these brief UI freezes.
I think it might be good to set some guidelines in the JUCE API doc on getStateInformation though, because I made the mistake of assuming that this is only called when a plugin is closed or a project is manually being saved.
Most hosts do an auto-save from time to time to minimize data loss when a crash happens. This is the reason why bitwig asks for the state every minute. Some hosts probably only ask when the plugin notifies the host about a parameter change. This is something we can’t control in the plugins.
A lot of plugin operations must be done in the main thread by specification to simplify things, avoid multi-threading issues, and keep complexity low. Just imagine the problems that can happen when the plugin sends parameter changes at the same time when the host asks for the state.
Unfortunately, this is also the UI thread. Stuff called from the main thread must be fast to ensure the UI is not blocking. It’s a simple design, but it does not scale.