Worker thread for audio processing


#1

Hope some of you kind experienced folk might be able to help me out. Been trying to debug this issue for a couple of days now. 

I have some real-time pitch detection going on in my app. I have a PitchTracker class which inherits from AudioIODeviceCallback. The audio callback is just collecting samples until a buffer size is reached and stops:

    void audioDeviceIOCallback (const float** inputChannelData, int numInputChannels,
                                float** outputChannelData, int numOutputChannels,
                                int numSamples) override

    {

        
        //const ScopedLock sl (audioCallbackLock);


        for (int i = 0; i < numSamples; ++i)
        {
            {
                float inputSample = inputChannelData[0][i]; //mono / left mic only

                if (currentSample < bufferSize)
                {
                    currentFloatSamples[currentSample++] = inputSample;
                }
            }
        }
    }


The same class has a calculate method which calls the pitch detection routine:

    void calculatePitch() noexcept
    {
        const ScopedLock sl (audioCallbackLock);

        if (currentSample < bufferSize) 
        {
            return;
        }
        else
        {

            float newPitch = pitchYin.getPitchInHz (currentFloatSamples);
            currentSample = 0;

            if (newPitch != currentPitch)
            {
                currentPitch.store (newPitch);
            }

        }

    }

currentPitch and currentSample are both wrapped in std::atomic. 

I have a WorkerThread, which calls the calculatePitch method when requested, set by a timer in a component:


class WorkerThread : public Thread
{
public:
    WorkerThread() : Thread ("WorkerThread"), getNewPitchValue (false)
    {
    }
    
    ~WorkerThread()
    {
        stopThread(-1);
    }
    
    void run()
    {
        while (true)
        {
            if (threadShouldExit())
                return;
            
            if (getNewPitchValue)
            {
                //const ScopedLock sl (calculateLock);
                pitchTracker->calculatePitch();
                
                getNewPitchValue = false;
            }
        }
    }
    
    std::atomic <bool> getNewPitchValue;
    
private:
    //CriticalSection calculateLock;
    
    SharedResourcePointer<PitchTracker> pitchTracker;
    
};
​

(As you can see I've been playing with CriticalSection / ScopedLock's)

My problem is that, about 75% of the time its as if there is no or very low input (gives the lowest possible frequency reading). Occasionally it gives a reading of 88200 hz (!). And about 25% of the time it tells me the pitch as expected. If I then pause and resume execution this then falls back to the the low/no input behaviour. 

Before I split the calculate method into the worker thread, I didnt have this problem, however I need a seperate thread to maintain a smooth GUI. 

I've done some analysis of the buffers by writing them to wav files. When it works, most of them show a clean section of sound. However some of them show a glitch which looks like the buffer didnt finish writing before it starts the calculate method. (i.e. two pieces of audio spliced together)

When it doesnt work the audio buffers are full of very low amplitude noise. 

I've tried both on OSX and iOS. 

I've got a strong feeling this is to do with thread safety and locks etc., a subject I'm not very experienced in. I thought I just need to lock the calculate method. Obviously something else is needed here. 


#2

What is the value of bufferSize and the timer interval ? To get enough resolution in bass frequencies, I guess that bufferSize / sampling rate >> timer interval, which explains why the pitch estimation returns 0 most of the time. What is wrong in your algorithm imha is that you have to specify some overlap. Instead of estimating the frequency from all new data all the time, you should do from buffers which contain some redundant data with a circular FIFO. This way, (bufferSize - overlap) / sampling rate << timer interval and you are good.

About the way you handle the worker thread, I will let other people give you their opinion since I'm not that confident in this area either.


#3

Thanks Wolfen, I have written a circular FIFO class to do just that. I had some problems with it and wanted to nail down the thread issue before getting back to that.  The pitch detection has been working fine just filling buffers end to end. My issue seems to be due to threading. To answer your questions: buffer size is 2048, timer is running at 30hz. 


#4

Well at 44100 Hz, the buffer needs 46.4 ms to be filled, and the timer callback happens every 33 ms. That means that without overlap in this particular case you might not update the pitch as much as needed, but anyway that shouldn't set the pitch to zero, so the problem might not be in the code you have provided...

For your thread, there is just one thing that is bothering me : what is the use of your critical section ? Do you use the lock in a function which is not displayed here ?

Last question : do you have memory leaks somewhere, which may be the reason your buffer has its content corrupted ?


#5

Well, the thread thing is obviously a red herring. I tested again without the thread running (calling the calculate function directly from the timer) and its still behaving the same way. Quite possible this is a memory leak thing, hence the random occurance. Thanks for the suggestion. Now to look at how to debug possible memory leaks.... 


#6

There's some really neat told for detecting memory leaks. I just started using Visual Leak Detector (Windows) and it works quite well. It shows the leaks detected in the debug output window and even lets me click on it to go to the leaking code. You just have to include a header file in one source file for it to work, I believe. 

http://vld.codeplex.com

There's another tool for Linux that's quite popular that can detect memory leaks too. I think it's called Valgrind with Memcheck. I think it actually works on OS X too. It's definitely worth checking out.

http://valgrind.org/docs/manual/mc-manual.html