GUI unresponsive even through CPU cores are not too occupied, likely I need multithreading. Where do I start to solve this?

[onar3d]

Hi!

I've written a ~30k line Juce program thus far with nearly no multithreading, and probably therein lies my problem.

I've read up on the forum and source documentation without reaching a conclusion on the best way to go so I thought I'd ask for some pointers:

Very briefly, the point of my program is to receive loads of OSC messages over UDP, modify them interactively based on continuous input from the GUI, and send the results out again ASAP as OSC.

I only use a thread in the UDP receiving, where new packets are communicated from a thread to the main program with PostMessage from the UDP receiving thread.

My model uses Value objects, which are either set internally responding to incoming data, and their changes reflected in the GUI following Value::Listener::valueChanged(...) messages, or set following GUI components calling Value::setValue(...), and then the model reacting to the above event.

When there's a little bit of activity all is fine, but when there's more, the GUI freezes entirely, and I know this is a problem in my programming rather than simply too much data since the same amount of processing in a JAVA & Swing prototype I had made works just fine.

Now I know multithreading is a big topic, but perhaps you can get me started by pointing to how this issue is solved in examples, or what the main tools in Juce are for this? For example, I've read that Value objects are not thread safe, how do I best deal with that?

Sorry if the question is too general, any direction to discussion/reading that can get me started is much appreciated!

 

[TomSwirly]

To start with, you can only do trivial amounts of work on the GUI thread - any actual calculation must be on a separate thread.

The fact that it works on Swing, on your machine, is the purest luck.  On another machine, it might skip or freeze.  (Or Swing, these days, might automatically give you a worker thread - I haven't used it in a decade.)

Threading is quite hard - I don't have a great reference.  This stackoverflow thread http://stackoverflow.com/questions/5125241/how-to-make-an-application-thread-safe isn't bad.

I make huge use of threads, but unfortunately none of the code is open-source.

[TomSwirly]

Very few classes are thread-safe by default - you should just assume that no class that isn't explicitly listed as thread-safe is not.

You must solve this with locking.  For each object that's used from multiple threads, you need to have a juce::CriticalSection.  Right before you start changing or reading the object, you must construct a new juce::Lock with that critical section (nearly always as a local variable).  Right after you finish changing or reading the object, you free the lock.  It nearly always looks like this:

class Foo {
 public:
  // ...
  int getCount() const {
    Lock l(lock_);
    return count_;
  }

  void setCount(int count) const {
    Lock l(lock_);
    count_ = count;
  }

 private:
  CriticalSection lock_;
  int count_;
};

[Peter_Emanuel_Roos]

I am running into the same issues.

Before I started learning Juce a few months ago, I experimented with WDL/OL (I especially liked its cross platform approach AND the inclusion of installer build options!). The few demo's and GPL plugins that I built with it are VERY responsive with their knobs and sliders. But WDL is not as active or well maintained like Juce!

The plugin that I am working on uses some 3% of my both cores in Reaper, yet the sliders are becoming more and more sloppy and slow as I add some basic functionality and audio processing. The Task Manager mentions that Reaper uses 40% of my CPU cores (because it uses a high priority process?).

I don't know where to look for my bottlenecks, but I do hope that someone can convince me that even with more heavy processing, Juce plugin GUIs can really remain snappy and responsive - and thus, that I and the OP are doing something wrong.

Hence my suggestion for a sticky on: Tips on keeping your GUI responsive.

Thanks in advance!

Peter

[jules]

Please don't get trapped into thinking that there's any kind of fundamental problem with juce - if your GUI is unresponsive, it's definitely the fault of your own code! All GUI toolkits + libraries work essentially same way, and none of them will work well if you write code that clogs-up the GUI thread.

Hence my suggestion for a sticky on: Tips on keeping your GUI responsive.

Tom's post already answered that. Sorry if it's pithy, but the only tip is "Don't run code on your GUI thread that blocks it!"

I'm afraid that there's really no shortcut to just properly learning about threading, and applying all the normal threading techniques to what you're building.

 

[mikeg]

I believe that the answer will be positive but I want to be sure...

Do the PluginEditor and PluginProcessor methods runs on separate threads?

As this is basically the most important thread separation that has to be provided in order to allow responsive GUI of a plugin, right?

So nothing to worry about that if you develop plugin with JUCE, correct? 

[Peter_Emanuel_Roos]

I already admitted that I am beginner, sorry if my post sounded like pointing at Juce.

Already learnt some more, fixed some stuff (indeed more beginner's errors), also by browsing the forum.

[jules]

Wish I had some good advice to help you! Threads are just inherently difficult!

[TomSwirly]

1. Never do anything that takes anything more than a very short time on the GUI thread.

2. If a variable can be read and written from multiple threads, always lock it during reads AND writes.

3. Never hold a lock for more than a very short time.

4. Deadlock elimination:  If you can, make sure that you never have two locks at one time.  If you can't, make sure that you always take the locks in the SAME order.

[onar3d]

Thank you for the help, I like the very simple and concise description of the issue here, it makes it seem far more manageable for sure!

Java's Swing definitely separates your "Main" thread from the GUI from the get go, when you create a GUI that is in its own thread automatically, and then you have to lock variables when accessing them.

Clearly for your replies, in Juce this is to be manually created by the programmer and not on by default.

I still have questions on the issue that are specific to how Juce expects you to deal with threading, but I will be posting these in separate forum post threads as to not clog this one up.

Thank you!

[Bruce_Wheaton]

Hopefully I can offer something more useful to your situation.

Quick fix: look at threadpools and jobs. See if they will do what you need. In that case, you would package up a task, and all the data it needs, into a job object, and send it off to get executed. There's a little work to do when the job is complete, but you can do it on the main thread.

Side note: although locking (critical sections) is essential, it's not a good policy as your main data sharing scheme. All that locking and unlocking has the effect of linking the multiple threads, eventually enough that you're back where you started.

A better alternative, janky though it may sound, is to have the same structures on each thread, and keep them synchronized. That means each thread has complete and local access to all the data it needs to do a job, without lots of locking and unlocking, which will gum up other threads.

So, in your case, you need a worker thread, and to send your data to that, not the main thread. That means you can't use post as your main cross-thread mechanism. It also means you can't use Values as your main store- a major flaw in values, that knocks 50% of the value of them, IMHO. What you can do is use values to store data and reflect gui.

So - you, like many of us, have to implement the missing piece of values - cross thread access. You already did that once, with messages. This time, you'll want to implement lock free FIFOs between each thread and the worker thread. One will have the incoming OSC. Another will be outgoing OSC. Another will be value changes, and another will be values that have changed on/from the GUI thread.

This will probably be overkill for your situation (OSC isn't much data compared to audio or video), but will do what you need. down the road.

 

Bruce

 

 

 

 

 

[TomSwirly]

> All that locking and unlocking has the effect of linking the multiple threads, eventually enough that you're back where you started.

I don't agree.  Locking and unlocking can work perfectly well IF you are reasonably careful.  Locks really aren't expensive - as long as you make sure not to keep anything locked for any period of time.

> It also means you can't use Values as your main store- a major flaw in values, that knocks 50% of the value of them, IMHO.

But how would you fix that - without locking?  They're too complex for the Atomic operations to work on them...

> This time, you'll want to implement lock free FIFOs between each thread and the worker thread. 

Implementing a correct lock-free FIFO queue is quite hard: http://www.drdobbs.com/parallel/writing-lock-free-code-a-corrected-queue/210604448.  I didn't find a JUCEy implementation, which you'd want to be platform-independent.

Also, FIFOs have their own downsides - the first one being that there needs to be some sort of thread to get data out of that FIFO (because you can't have whatever thread pushes onto the queue also notifying the other thread, and potentially blocking...)

Unfortunately, as I keep mentioning, there's a potential race condition in Juce's notify system so you can't just wait indefinitely and get notified by the queue updating - you need to spin at least somewhat.

(What's the race condition?  If you have a thread reading a queue, you wake up, check if there's anything on the queue, and then go back to sleep.  Unfortunately, a notification might come in between the time you check the queue and the time you go back to sleep - so you can only sleep for a short time just in case...)

[Bruce_Wheaton]

OP - feel free to ignore this...

>> All that locking and unlocking has the effect of linking the multiple threads, eventually enough that you're back where you started.

>I don't agree.  Locking and unlocking can work perfectly well IF you are reasonably careful.  Locks really aren't expensive - as long as you >make sure not to keep anything locked for any period of time.

Locks create links between the threads. If you have a high priority thread it can prevent a lower priority thread, such as the message thread getting anything done. No way around that - it's not relevant whether the actual locking is cheap, it's what work is being done. It's a dead-end.

>> It also means you can't use Values as your main store- a major flaw in values, that knocks 50% of the value of them, IMHO.

>But how would you fix that - without locking?  They're too complex for the Atomic operations to work on them...

You read the next line of my post?

>> This time, you'll want to implement lock free FIFOs between each thread and the worker thread. 

>Implementing a correct lock-free FIFO queue is quite hard: http://www.drdobbs.com/parallel/writing-lock-free-code-a-corrected-queue....  I >didn't find a JUCEy implementation, which you'd want to be platform-independent.

Here's something for you then: http://bit.ly/1ibHRIw 

It's been there for three or four years, and appears to be rock-solid. I use it a few ways, including using reference counted XML payloads.

>Also, FIFOs have their own downsides - the first one being that there needs to be some sort of thread to get data out of that FIFO (because >you can't have whatever thread pushes onto the queue also notifying the other thread, and potentially blocking...)

That's true. You do have to have a running thread to check the queue, but both the message thread and any 'worker' threads will be running. I view that as an advantage - it makes updates 'modular' in that you can choose when to pick up new data. For instance, if the OP has a set of incoming OSC, they could collect all the derived changes so that all the results happen at once, and then process them all in one block too.

 

[TomSwirly]

First, I agree with you that OP should probably just use a lock-free queue, the end - particularly since I now know that Juce has one built in!  Queues just work, and they're a "one-stop shopping" solution.  I probably should have prefaced 

I didn't find the Juce FIFO in my searches - I searched for "Juce lock-free queue" and similar queries, where that result doesn't appear at all.  Perhaps this is a difference in terminology - I would call a first-in, first-out structure a queue, not a "FIFO", which is an adjective and not a noun for me! (and I note both the STL and Python agree with me on this...)

>>> It also means you can't use Values as your main store- a major flaw in values, that knocks 50% of the value of them, IMHO.

>>But how would you fix that - without locking?  They're too complex for the Atomic operations to work on them...

>You read the next line of my post?

>>> This time, you'll want to implement lock free FIFOs between each thread and the worker thread. 

Yes, I did.  That isn't going to fix the class Value, though!  If you design your system to use a queue entirely, then you won't have any locking issues - but that isn't "fixing" Value at all.

Queues are a one-stop shopping solution, yes - but locks work better in some cases.

Using a queue can definitely make things less zippy.  As an example, I originally had a queue driving my millisecond digital time clock in my music application.  But in order to get the most accurate update, you need to update the time every sample buffer.  When I had a queue, I was unable to get a completely smooth clock (though I'm very picky) - when I moved to a variable shared between the threads and a lock, I was able to get it running completely smoothly, even in an unoptimized build on an old machine.

And, sometimes you just want a getter for a variable that's sometimes set on a different thread than read.  Creating a queue for just this purpose would be overkill, particularly if that value only changes rarely.

[onar3d]

Just to thank Bruce and Tom!

Your replies and discussion here helped me make my program workable again :)

I ended up with a hybrid for now, with a FIFO between incoming OSC and worker thread, and locking between worker and GUI.

That means whenever there's updating of the GUI, the incoming OSC messages pile up if they are very many, because I have to use a MessageManagerLock in my custom ValueSource's setValue and getValue. But for now it's OK.

That too can be solved, just that I need to do more refactoring, so that I don't rely on Value to communicate changes between GUI and worker threads, and instead use some other mechanism that involves lock-free FIFO's as Bruce suggested.

Thanks again!

[TomSwirly]

We aim to please!  :-)