Timur Doumler Talks on C++ Audio (Sharing data across threads)

You can do this with a single queue - just have the main thread do the deallocation when it needs to ‘overwrite’ a position on the queue. Not a great solution if you’re posting giant objects around, but works nicely for small command objects.

  • one from audio thread to main thread to deallocate memory, once the object is no longer needed by the audio thread.

The problem in this direction is: What do you do on the audio thread if the queue is full? You can’t allocate more space for it, and you can’t wait for the main thread to dequeue more items.

You can do this with a single queue - just have the main thread do the deallocation when it needs to ‘overwrite’ a position on the queue.

How do you ensure that the object you’re deleting on the main thread isn’t currently being used by the audio thread?

How do you ensure that the object you’re deleting on the main thread isn’t currently being used by the audio thread?

My guess is with a reference count. Although a lot of these problems are solved by using IDs/Generations for objects instead of pointers, so all the data is preallocated using an arena or something and all you need to do is pass integers to the audio thread.

1 Like

The idea is that the audio thread takes ownership of whatever data it’s using. So, if there’s some object that’s been allocated on the main thread, it’ll be passed over as part of a command object, and then the audio thread can take ownership of that collection by swaping it into place. This is fast + safe as long as the swap doesn’t allocate and is noexcept. Neither thread should keep around pointers to collections that they don’t own any more, and if a collection is in the queue, neither thread owns it.

which thread owns the queue??

Which thread owns the AudioProcessor?

are there 3 threads in total?
AudioProcessor
Main Thread (GUI)
some other thread that owns the queue, which the collection resides in?

I’m saying that the queue would probably live inside the audio processor somewhere, and your editor interacts with the processor (ideally) just by posting messages into the queue. But while an object is in the queue, neither the gui nor the processor should know about it or have pointers to it.

2 Likes

ah ok. that makes sense.

@Holy_City If I understand correctly what you’re aiming for, using a reference count means that it might become 0 on the audio thread, which would cause a heap deallocation. I agree that some other solution is needed, like arenas or a manager with IDs.

@reuk If the audio thread takes ownership of whatever data it’s using, how does it release the ownership of data it doesn’t need anymore, without calling delete?
For example, if you call swap on the audio thread to swap a command object, then the command object can’t deallocate in its destructor. But if it holds an object that was allocated on the main thread as you said, it has to deallocate that somehow.
Maybe I’m missing something in your explanation though. I think this is an interesting problem and actually not simple to solve in a “perfect” and general way.

If I understand correctly what you’re aiming for, using a reference count means that it might become 0 on the audio thread, which would cause a heap deallocation.

Not really. If the main thread creates the object, it has 1 reference, then if it goes to 2 it’s “live” on the audio thread, audio thread drops it and now it’s one so the main thread knows it’s the only owner and alright to delete it. Which is kind of similar to how generation tricks work, where it’s like reference counting as a state machine.

1 Like

Ah ok, so it’s a similar idea to Timur’s ReleasePool.

I use exactly that setup in my VideoEngine, because here the AudioThread, the GUI and the Video composer are all reading from the same clip.

And the auto release pool serves here the double purpose, since some of the clips have a background thread (TimeSliceThreadClient), so when the reference count drops to 1, it is removed from the TimeSliceThread and destroyed on a non realtime thread.

Also the shared_ptr has the advantage over ReferenceCountedObject, since I can store a weak_ptr in the GUI, so if it is removed in the model, the GUI will not hang on to it (but extend it’s lifetime while being used calling weak_ptr.lock()

1 Like

It swaps ownership with the object in the queue. So, the audio thread ends up owning the object that it popped off the queue, but now the queue owns the old/expired object that the audio processor was last using. This expired object won’t be destructed until the main thread overwrites that position in the queue.

1 Like

Sorry, had to read up…

So the audio thread puts it back into the queue, but in a slot that is known to be expired, so it can be swapped from a new message, and the swap happens from the message thread?

The queue is a lock-free circular buffer. Adding an item to the queue only happens on the main thread. When adding an item, you overwrite the object that used to occupy that position in the buffer, which frees that object. When removing an item from the queue, you do a swap, so after the swap, the ‘reader’ owns the item that was at that queue position, and that queue position is occupied by the reader’s last state.

2 Likes

Aaah, I didn’t understand the swap bit… now that’s clear, thanks!

@reuk Thanks for the explanation :+1: An interesting way to do it.

Is this the same thing you’re doing in that Audio Sampler demo plugin with that CommandFifo usage?

No, that has an incoming and and outgoing queue.