Latency build up on Raspberry Pi


#1

I’ve had an issue reported by a client for whom I’ve built a simple audio recording and playback app running on a Raspberry Pi. The app is designed to run on a kiosk and will therefore be running for days/weeks on end. He’s reported an issue where, after the app has been left running for a while, there is a build up of latency during recording which results in silence at the beginning and audio cut off at the end (its a fixed length recording). The longer its left running the bigger latency. I wonder if this is something anyone else has come across? There isn’t anything immediately obvious in my code that might be causing this.


#2

I believe this could happen if the audio thread is not always able to keep up, e.g. the audio callback does not return quickly enough for the I/O buffer size. Perhaps your audio thread does some non-realtime safe calls, like assigning memory to the heap (yes, that’s disallowed) or locking onto the Message Thread.
I suggest running a profiler to see if you can find any hidden mallocs on the audio thread.

Here’s a good read on realtime-safety: http://www.rossbencina.com/code/real-time-audio-programming-101-time-waits-for-nothing

As a workaround I suggest resetting the audio I/O device every now and then (perhaps after each recording?), that could probably help with the latency.


#3

Thanks, I’m now only starting the device before recording or playback and closing after. Hopefully that solves the issue.

The audio thread in this app is super simple, its just copying the audio to a pre-allocated buffer, which is then saved to a file via AsyncUpdater. I did recently read here that triggerAsyncUpdate is also not safe to call on the audio thread, but I don’t think that would be causing the issues I’m seeing…


#4

Communicating a boolean flag (e.g. dirty state) in a realtime- and thread-safe manner

Writing to the message loop is blocking, as I had to learn the hard way.
If you only need to notify another thread that a buffer should be saved to disk, you can easily set a std::atomic_flag in the audio thread, which is guaranteed to be lock-free by the C++ standard, and frequently consume it from the other thread.

Note that you have to use the atomic_flag to indicate the opposite state, as you can only clear (set to false) and test_and_set it (consume state and set to true).

Therefore, you could do:

// cleanFlag is a member variable of type std::atomic_flag
// to mark the flag dirty, e.g. from the audio thread when the buffer needs to be saved
cleanFlag.clear();
// cleanFlag is now set to false, indicating the buffer is dirty

// consume the flag in the other thread, setting it to true.
// you would do this in a loop (e.g. from a juce::Timer), 
// as you don't get a notification callback from an AsyncUpdater.
bool isDirty = !cleanFlag.test_and_set();
if (isDirty) {
      // thread-safely access the buffer and write it to a file.
}

Communicating complex data structures in a realtime- and thread-safe manner

If you need the audio thread to continue processing while saving the file, you need to communicate the buffer to the file-saving thread in a thread-safe and non-blocking manner (mutexes are not okay, as waiting for lower-priority threads will cause priority inversion!).
To achieve this, I suggest a lock-free queue holding up to 1 instance of your buffer.
The audio thread can safely write to the queue by adding the buffer to the queue once it should be written, and the file-saving thread periodically polls the queue and saves the buffer object to a file when it receives one.

I can only recommend moodycamel’s ReaderWriterQueue as the lock-free queue implementation of my choice.

Example code:

// Your buffer object. 
// Must have a copy assignment operator to work with the queue
struct MyBufferObject {
    static const int size = 4096;
    std::array<float, size> data;
}

// The queue is a class member of an object both threads can access, 
// e.g. the AudioProcessor
moodycamel::ReaderWriterQueue<MyBufferObject> bufferQueue(1);

// ==========================================
// writing to the queue in the audio thread:
// ==========================================

// use try_emplace to ensure the queue never grows (which would allocate memory).
// if the file-saving thread does not fetch the queue frequently enough,
// the object is lost.
bufferQueue.try_emplace(bufferObjectInstance);


// =======================================================
// reading from the queue in the file-saving thread loop:
// =======================================================

// targetBufferObject is a class member of your file-saving thread, 
// so the array is not allocated for each loop invocation
MyBufferObject targetBufferObject; 

// read from the queue if it contains an object, if not, return
if (!bufferQueue.try_dequeue(targetBufferObject)) return;
// do something with the targetBufferObject, like saving it to a file

In the lock-free queue scenario, you obviously wouldn’t need the atomic_flag.
Of course, copy-assigning the buffer object into the queue should be a quick operation, so the audio thread is not busy for too long when calling try_emplace. Depending on your buffer size, transferring the whole buffer at once may not be suitable.

Conclusion

That being said, if you can afford restarting the device after saving the file, you can easily share the buffer object between threads and protect it using a mutex, as long as there’s no realtime audio that must be processed at the time of accessing the buffer from the file-saving thread.
In that case, I’d recommend the lock-free std::atomic_flag approach. You may also use an std::atomic<bool>, of course, which is lock-free on most architectures, I guess, but given that you’re on a Raspberry Pi, I’d prefer the atomic_flag, which is guaranteed to be lock-free.

Anyway, this should serve as a general guide on how to safely communicate between a realtime thread and other threads, maintaining both realtime- and thread-safety. Your implementation will not need all of that complexity, if you can simply stop the realtime thread.

Hope this helps.
Best,
Marius