Hello all. Thanks in advance for any help or advice
I’m seeing that in the juce_AudioProcessor.cpp in the function AudioProcessorParameter::sendValueChangedMessageToListeners there is a ScopedLock lock (listenerLock)
I’m having a hard time understanding how that works since that function is called (presumably) from the audio thread when there’s some automation going. I see that the other places where that lock is acquired is when a listener is added to the listeners vector. From what I understand SliderAttachments register themselves as listeners upon creation but I feel like I’m missing something in all this. So my question is: Wouldn’t that imply that opening or closing the GUI window could potentially block the audio thread if automation is happening at the same time?
This lock is to avoid changing the listener list while a parameter update is iterating the listeners to notify them. If you would allow that, the iterators are invalidated and you are facing UB including a potential crash.
The host and the wrappers will call that function from all kind of paths including the audio thread. So it is not the users choice to not call it.
There needs to be a decoupling, IIRC the AudioProcessorValueTreeState was aiming to solve that, but suffered the same problem with an AsyncUpdater.
I haven’t followed all paths to the end, so maybe there is something I am missing, why it is safe. But like the OP I would expect potential glitches when the UI is opened or closed, although very rarely.
This lock is to avoid changing the listener list while a parameter update is iterating the listeners to notify them. If you would allow that, the iterators are invalidated and you are facing UB including a potential crash.
Yeah I think I understand why the lock is necessary.
The host and the wrappers will call that function from all kind of paths including the audio thread. So it is not the users choice to not call it.
I think that’s sort of my problem, apparently that would mean that I can’t ensure the audio thread is lock free unless I modify the JUCE Framework, right?
I haven’t followed all paths to the end, so maybe there is something I am missing, why it is safe. But like the OP I would expect potential glitches when the UI is opened or closed, although very rarely.
I think this tends not to happen because parameters are not usually changing when the UI is opening (unless there’s some automation running) and also that the lock is avoided entirely if a notification is produced but the newValue is equal to the oldValue because in that case no listener is notified. At the same time I was under the impression that just creating a bunch of GUI elements like new tabBarComponents and stuff and registering them to the AudioProcessorValueTreeState was sort of safe but what Im understanding is that it is actually something that could lock the audio thread!
Final question if you all don’t mind. I’m receiving some control data via OSC to change an audio parameter. Most of the times all of this is happening in the message thread but once in a while, when notifying the host about one of this changes I get a call from the audioThread to that AudioProcessorParameter::sendValueChangedMessageToListeners through processParameterChanges in juce_audio_plugin_client_VST3.cpp .
Am I right in assuming that it is always the case that there’s a call to this function from the audio thread? it just so happens that most of the times the message thread writes the value first and then the comparison of the proposed value by the audio thread and the value present in the parameter is equal so it returns early without getting to that lock? I dont quite understand why that never happens with a slider attachment dragged with a mouse but happens with an OSCListener driving the changes O.o
A lock is no problem when it is not contested. So if you don’t attach listeners to the parameter, it is still lock free.
There are alternatives, but each comes with a different cost. So each one has to pick their poison.
I recommend watching @dave96 's great talk on that topic:
The juce code can be improved, you can work around the problems or you can roll your own. But a perfect “one size fits all” is unfortunately not available.
Thanks daniel, you are right, I got everything mixed up in my post. There is an observation that I think it still holds though.
Even if listenerLock is intended to avoid changes in the listener list, sendValueChangedMessageToListeners (and ultimately setValueNotifyingHost, from where it is being called) can lock itself if it is called simultaneously from different threads.
As a result I’m going to assume that hosts normally update parameters from the message thread because this is where most plugins update their parameters. So probably it is not a good idea to update an automatable parameter (and call setValueNotifyingHost) from the audio thread.
On the contrary.
Host automation has to be in sync with the audio, hence parameters are set immediately before each processBlock() call. Some have a bespoke thread, but usually it is the audio thread.
It has to be that way, because the message thread would be way too slow in an offline bounce situation.
The message thread comes into play, when the user changes the parameter from the plugin editor. This can only occur in a real time situation, and it cannot be avoided that the GUI reaction is not synchronised with the audio on a sample level.
This is an ongoing discussion and afaik there is no proper solution (I haven’t read the proposal of the last post):
Parameter changes are already not accurate on a sample level inside plugins so I wouldn’t be surprised if some hosts relax that. However, yes, my bad again, the message thread is too slow so probably is going to be another thread (I still think the audio thread is a risky approach even for the host). The OP in the last thread that you mentioned for example talks about using a HighResolutionTimer effectively for automation in his plugin.
In any case, this is something that I should test in practice with different hosts. That would put an end to my (bad) assumptions
We probably all agree though on the fact that calling setValueNotifyingHost in a plugin from the audio thread is very risky because of that lock (and other things that the host may be doing).
I was thinking though that JUCE could separate the call to the host to remove the listenerLock.
Is there a way that JUCE adds the host as a listener in the constructor and already runs the rest of the (variable) listener calls in a separate asynchronous thread?
Would something like this make sense?
void AudioProcessorParameter::sendValueChangedMessageToListeners (float newValue)
{
host->parameterValueChanged(getParameterIndex(), newValue);
MessageManager::callAsync([this]()
{
ScopedLock lock (listenerLock);
for (int i = listeners.size(); --i >= 0;)
if (auto* l = listeners [i])
l->parameterValueChanged (getParameterIndex(), newValue);
});
}
I’m saying this because the other listeners are generally running in a separate asynchronous thread or the message thread anyway…
There are two problems with that:
a) some designs might rely on a synchronous callback
b) callAsync is also not realtime safe thanks to a potential lock in the postMessage buried inside the callAsync.
a) In that case, at least there should be a flag to skip the lock for those who don’t want to go down that road…
b) Oh wow, I didn’t know that. If that’s the case this is terrible. I would expect a lock-free callAsync implementation. Isn’t that possible?
I think the point of the original question is that there doesn’t seem to be a way to avoid locks altogether. The lock that I was referring to originally is in AudioProcessorParameter::sendValueChangedMessageToListeners and that function is called both when a slider or GUI element changes a parameter (even if there’s no AudioProcessorValueTreeState) and when the host produces automation information. The lock is there to protect the listeners list so that no listener can be added or removed while we are transversing the list to notify the listeners.
In the first case (when called from the message thread) it needs to be there because it happens because a GUI action initiated the call. In the second case it needs to be in the audio thread because otherwise there’s no way to produce sample accurate automation.
note that:
although many plugins do not aim to guarantee sample accurate automation, that feature was a big one for VST3 in general (its like their 3rd item on the about VST3 in their GitHub)
3. Sample-accurate Automation
VST 3 also features vastly improved parameter automation with sample accuracy and support for ramped automation data, allowing completely accurate and rapid parameter automation changes.
the only way to achieve that is to put a list of parameter changes IN the audio callback and stamp them by a sample offset so that they can (potentially) be applied exactly at that sample. Because of this that call to notify the listeners happens on the audio thread and can’t be done in any other way.
That call to notify the listeners is actually happening every process call and is something that is in the framework no matter how you write your plugin. If you add a parameter to your plugin, each process call there will be a call in the VST3 wrapper that tries to notify the listeners. Most of the times it doesn’t go through the lock because there are no parameter changes to notify so no notification happens effectively, or because the parameter was already updated so in the process call the “newValue” is already the one in the parameter so the notification is also skipped. Moreover, even if it goes through the lock, that lock is rarely contested so in theory it shouldn’t be a problem (or that is the logic from what I understand)
Because of 1 & 2, this is NOT about how to write the plugin itself, it is a consequence of how the JUCE code interacts with the VST3 wrapper that looks strange from the perspective of a JUCE user because JUCE itself doesn’t support sample accurate automation
Personally I’m more inclined to think that the solution doesn’t have to do with how to notify the listeners of the parameter changes but with removing the need for the lock in the first place, but that can only be done if adding or removing listeners cannot conflict with transversing them.
ultimately, JUCE needs to support sample-accurate-parameters, because this feature pretty much forces JUCE to adopt a cleaner ‘separation of concerns’ between the various objects and threads.
