Implementation of InternalMessageQueue still problematic

The InternalMessageQueue now collects messages in a overflow queue as soon as PostMessage() returns ERROR_NOT_ENOUGH_QUOTA.

However this still means that JUCE maxed out the Windows messages queue leaving no space for further messages for anyone to post. At least some hosts such as FL Studio also seem to post messages while creating a plugin which then fails. The hosts do not seem to have a fallback mechanism (which is understandable because why would thousands of messages be queued under normal circumstances) and as such they can end up in a broken state.

I did a quick hack that fixed the issues and I suggest to implement something similar in the official codebase: As soon as 1000 messages have been send using PostMessage() all further messages are put into the overflow queue until the overflow queue is being dispatched.

To me this seems like it would be fixing a symptom and not the actual cause, because if a plug-in is maxing out the message queue limit on Windows then we should look at why this is happening instead of patching the issue. When I was testing this previously, I was using a pretty contrived example to flood the queue so I’d be interested to see what your plug-in which is causing this is doing. If it’s a fairly simple JUCE plug-in with nothing special then we should look at what might be happening in JUCE that could be improved.

Our plugin UI has a lot of controls. All the controls were being created when the plugin UI was created. This was fine for a single instance of the plugin, however if you loaded a saved session that had several instances of the plugin with the UIs open, each UI would get created at the same time and flood the message queue.

Our solution for now is to not create controls until the component for the specific layer is visited by the user for the first time. This is working for us for now, but a more general solution would be prefered.

For example, a juce::TextEditor posts 4 messages from its constructor. The message queue has a depth of 10000 messages. So let’s say our UI has 256 text editors (8 layers x 32 text editors per layer) and the user opens a sessions with 10 copies of the plugin then we are sending 4 * 256 * 10 = 10240 messages and the message queue overflows. I think that’s a pretty ‘normal’ use case to run into the issue.

Since the message queue is shared between between all windows created by the same thread (DAW windows and plugin windows) there may actually be less room in the queue since other windows will be generating messages as well.

I think a better design would be to always use the overflowQueue and to only PostMessage when the queue is transitioning from the empty state to 1 item. And when handling the message, flush the entire overflowQueue. This is more similar to the macOS implementation which I’ve never had any issues with.

It actually is not really working anymore. FL Studio is also creating Windows messages and with enough instances of Nexus3, loading a project in FL studio becomes impossible. FL studio simply freezes, probably waiting for some message that never arrives.

Since ImageLine is listed in the “using Juce” area of the website, is it safe to assume it uses the message system? Because I doubt they use the latest develop version with the fixed overflow queue, so once they post a message and the queue is already full, it gets lost.

Juce needs a better message system as described by G-Mon.

You also have to consider that people load multiple instances of the same plugin, thus compounding the problem. Since Nexus3 is so light on the CPU it’s not unheard of that people use 20-30 instances of it. Not all of them are necessarily playing at the same time, but they get loaded nonetheless. Now add other plugins or the host using Windows messages and you’ve got a perfect storm of fighting over a limited resource with no good recovery method.

For now we forked JUCE and implemented the message handling the same way it’s implemented for Mac and it works just fine.

Would be great if you can share it as a squashed patch.

Here it is:

windows_bug.diff (4.9 KB)

I’ve tested with 22 instances in FL Studio now and all problems are gone.

1 Like

Thanks, we’ll take a look next week and do some testing and hopefully get something along those lines merged in.

OK, we’ve just pushed the following commit which should fix this issue:

Thanks for the help on this!

2 Likes

Awesome, thanks.