Sample drops in analyzer

gui
audio
dsp
#1

I’ve been working on an analyzer app, but I’m getting sample drops, and that introduces noise in the visualization making it barely usable.

Here is an example of a minimal implementation of an oscilloscope, using some code from the processing audio inputs and the dsp tutorial for the abstract fifo implementation and plotting:

#include "../JuceLibraryCode/JuceHeader.h"
#include <array>
#pragma once

//==============================================================================
class MainContentComponent   : public AudioAppComponent, private Timer
{
public:
    //==============================================================================
    MainContentComponent()
    {
        setSize (600, 100);
        setAudioChannels (2, 2);
		startTimerHz(30);
    }

    ~MainContentComponent()
    {
        shutdownAudio();
    }

	void prepareToPlay(int, double) override { audioProcessLoadMesurer.reset(); }

    void getNextAudioBlock (const AudioSourceChannelInfo& bufferToFill) override
    {
        auto* device = deviceManager.getCurrentAudioDevice();
        auto activeInputChannels  = device->getActiveInputChannels();
        auto activeOutputChannels = device->getActiveOutputChannels();
        auto maxInputChannels  = activeInputChannels .getHighestBit() + 1;
        auto maxOutputChannels = activeOutputChannels.getHighestBit() + 1;

        for (auto channel = 0; channel < 1; ++channel)
        {
            if ((! activeOutputChannels[channel]) || maxInputChannels == 0)
            {
                bufferToFill.buffer->clear (channel, bufferToFill.startSample, bufferToFill.numSamples);
            }
            else
            {
                auto actualInputChannel = channel % maxInputChannels; // [1]

                if (! activeInputChannels[channel]) // [2]
                {
                    bufferToFill.buffer->clear (channel, bufferToFill.startSample, bufferToFill.numSamples);
                }
                else // [3]
                {
                    auto* inBuffer = bufferToFill.buffer->getReadPointer (actualInputChannel,
                                                                          bufferToFill.startSample);
                    auto* outBuffer = bufferToFill.buffer->getWritePointer (channel, bufferToFill.startSample);
					auto count = 0; // counter of dropped samples

					scopeDataCollector.process(inBuffer, bufferToFill.numSamples);

					for (auto sample = 0; sample < bufferToFill.numSamples; ++sample)
					{

						if (inBuffer[sample - 1] == inBuffer[sample] && inBuffer[sample] == 0.0f)
							count++;
					}
				
					if (count>1)
						DBG("Dropped samples = " << count << ", buffer size: " << bufferToFill.numSamples << ", xRun: " << audioProcessLoadMesurer.getXRunCount());

					for (auto sample = 0; sample < bufferToFill.numSamples; ++sample) 
					{
						outBuffer[sample] = 0.0f;
					}
                }
            }
        }
    }
	//==============================================================================
	void paint(Graphics& g) override
	{
		g.fillAll(juce::Colours::black);
		g.setColour(juce::Colours::white);

		auto area = getLocalBounds();
		auto h = (float)area.getHeight();
		auto w = (float)area.getWidth();

		// Oscilloscope
		auto scopeRect = Rectangle<float>{ float(0), float(0), w, h / 2 };
		plot(sampleData.data(), sampleData.size(), g, scopeRect, float(1), h / 4);
	}

	//==============================================================================
	static void plot(const float* data,
		size_t numSamples,
		Graphics& g,
		juce::Rectangle<float> rect,
		float scaler = float(1),
		float offset = float(0))
	{
		auto w = rect.getWidth();
		auto h = rect.getHeight();
		auto right = rect.getRight();

		auto center = rect.getBottom() - offset;
		auto gain = h * scaler;

		for (size_t i = 1; i < numSamples; ++i)
			g.drawLine({ jmap(float(i - 1), float(0), float(numSamples - 1), float(right - w), float(right)),
						  center - gain * data[i - 1],
						  jmap(float(i), float(0), float(numSamples - 1), float(right - w), float(right)),
						  center - gain * data[i] });
	}

    void releaseResources() override {}

    void resized() override {}

private:
	template <typename SampleType>
	class AudioBufferQueue
	{
	public:
		//==============================================================================
		static constexpr size_t order = 14;
		static constexpr size_t bufferSize = 1U << order;
		static constexpr size_t numBuffers = 5;

		//==============================================================================
		void push(const SampleType* dataToPush, size_t numSamples)
		{
			jassert(numSamples <= bufferSize);

			int start1, size1, start2, size2;
			abstractFifo.prepareToWrite(1, start1, size1, start2, size2);

			jassert(size1 <= 1);
			jassert(size2 == 0);

			if (size1 > 0)
				FloatVectorOperations::copy(buffer[(size_t) start1].data(), dataToPush, (int) jmin(bufferSize, numSamples));

			abstractFifo.finishedWrite(size1);
		}

		//==============================================================================
		void pop(SampleType* outputBuffer)
		{
			int start1, size1, start2, size2;
			abstractFifo.prepareToRead(1, start1, size1, start2, size2);

			jassert(size1 <= 1);
			jassert(size2 == 0);

			if (size1 > 0)
				FloatVectorOperations::copy(outputBuffer, buffer[(size_t) start1].data(), (int) bufferSize);

			abstractFifo.finishedRead(size1);
		}

	private:
		//==============================================================================
		AbstractFifo abstractFifo{ numBuffers };
		std::array<std::array<SampleType, bufferSize>, numBuffers> buffer;
	};

	template <typename SampleType>
	class ScopeDataCollector
	{
	public:
		ScopeDataCollector(AudioBufferQueue<SampleType>& queueToUse)
			: audioBufferQueue(queueToUse)
		{}

		void process(const SampleType* data, size_t numSamples)
		{
			size_t index = 0;

			if (state == State::waitingForTrigger)
			{
				while (index++ < numSamples)
				{
					auto currentSample = *data++;

					if (currentSample >= triggerLevel && prevSample < triggerLevel)
					{
						numCollected = 0;
						state = State::collecting;
						break;
					}

					prevSample = currentSample;
				}
			}

			if (state == State::collecting)
			{
				while (index++ < numSamples)
				{
					buffer[numCollected++] = *data++;

					if (numCollected == buffer.size())
					{
						audioBufferQueue.push(buffer.data(), buffer.size());
						state = State::waitingForTrigger;
						prevSample = SampleType(100);
						break;
					}
				}
			}
		}

	private:
		//==============================================================================
		AudioBufferQueue<SampleType>& audioBufferQueue;
		std::array<SampleType, AudioBufferQueue<SampleType>::bufferSize> buffer;
		size_t numCollected=0;
		SampleType prevSample = SampleType(100);

		static constexpr auto triggerLevel = SampleType(0.05);

		enum class State { waitingForTrigger, collecting } state{ State::waitingForTrigger };
	};
	void timerCallback() override
	{
		audioBufferQueue.pop(sampleData.data());

		repaint();
	}
	AudioBufferQueue<float> audioBufferQueue;
	ScopeDataCollector<float> scopeDataCollector{ audioBufferQueue };
	std::array<float, AudioBufferQueue<float>::bufferSize> sampleData;
	AudioProcessLoadMeasurer audioProcessLoadMesurer;

    JUCE_DECLARE_NON_COPYABLE_WITH_LEAK_DETECTOR (MainContentComponent)
};

To use it I generate a signal with an external app and redirect the output to the input with a virtual cable (virtual audio cable on windows).
I’m using wasapi to be able to share the driver between apps using the internal soundcard, but you can also try it with Asio and an external souncard with a physical loopback cable.
In my app I have my own signal generator but I’ve not included any of that for simplicity.

This is how it looks:

image

I don’t think they are caused by concurrency problems, the problem seems to happen in the buffer given by getNextAudioBlock.
I am streaming an output message to the debug console there (I know this is a no no, it is only for debugging purposes to show the problem).
In the message I also show if there is an Xrun using AudioProcessLoadMeasurer but it is always zero, although I don’t know if I am using it correctly.

image

This happens very randomly and it is difficult to see it in a minimal example like this, although it deffinitively can be seen with patience, and in a more complex app they are more frequent.

Any idea of what can be the cause of this?

Thanks in advance!

#2

Only had a quick look, but what happens if the number of samples written is less than buffersize, looks like you maybe read more samples than are written?

Either you want to write the number of samples written with each block in case the block is smaller than the buffer size so you can return the right sized block from pop(). Or, my preferred method, you want to just use a block of memory, instead of lots of separate arrays, and write audio with the channels interleaved to your fifo which will make the fifo output independent of the size of the original blocks?

#3

Hi jimc, thanks for your reply.
In fact it is writing all samples until buffersize

while (index++ < numSamples)
				{
					buffer[numCollected++] = *data++;

, so if the block comes with some samples at zero, I can see them inspecting the collector buffer or the one in the abstract fifo.

Either you want to write the number of samples written with each block in case the block is smaller than the buffer size so you can return the right sized block from pop()

How can I know that? I am counting it this way

auto count = 0; // counter of dropped samples

					for (auto sample = 0; sample < bufferToFill.numSamples; ++sample)
					{

						if (inBuffer[sample - 1] == inBuffer[sample] && inBuffer[sample] == 0.0f)
							count++;
					}
				
					if (count>1)
						DBG("Dropped samples = " << count << ", buffer size: " << bufferToFill.numSamples << ", xRun: " << audioProcessLoadMesurer.getXRunCount());

But I don’t think this will always be accurate.

you want to just use a block of memory, instead of lots of separate arrays, and write audio with the channels interleaved to your fifo which will make the fifo output independent of the size of the original blocks?

Does this will solve this particular problem?
I’m going to investigate that for sure,

many thanks for your help!!

#4

Check the bufferToFill.numSamples is the same each time. It may be in which case it’s not a problem, but I don’t think it’s guaranteed.

#5

Hi Jimc, thanks for your reply.

I am taking into account bufferToFill.numSamples when collecting and writing the samples in the abstract fifo,
but I think my problem is that I’m getting less samples (or at least, less non-zero samples) than the reported by bufferToFill.numSamples.

You can see that in the previous image or in the above snippet. In the image bufffer size is in fact bufferToFill.NumSamples and I’m getting 292 samples of silence. And they not come from the signal generator for sure.

image

Do you think this is normal or could be a bug?

#6

A little update on that:
I tried among other things suggested, to do the same in a plugin, in order to discard differences between getNextAudioBlock and processBlock, but the results are the same.
So it may be a concurrency problem indeed.
This is the updated code replacing the normal timer by a high resolution timer as suggested in other posts, but the problem still remains.

I don’t know if the fact that I’m reading and writing to the buffer sequentially may affect. It is on the same thread but who knows…

#include "../JuceLibraryCode/JuceHeader.h"
#include <array>
#pragma once

//==============================================================================
class MainContentComponent   : public AudioAppComponent//, private Timer
{
public:
    //==============================================================================
    MainContentComponent()
    {
        setSize (600, 100);
        setAudioChannels (2, 2);
		//startTimerHz(30);
		videoThread = std::make_unique<VideoThread>(*this, audioBufferQueue, sampleData);
    }

    ~MainContentComponent()
    {
        shutdownAudio();
    }

	void prepareToPlay(int, double) override {}

    void getNextAudioBlock (const AudioSourceChannelInfo& bufferToFill) override
    {
        auto* device = deviceManager.getCurrentAudioDevice();
        auto activeInputChannels  = device->getActiveInputChannels();
        auto activeOutputChannels = device->getActiveOutputChannels();
        auto maxInputChannels  = activeInputChannels .getHighestBit() + 1;
        auto maxOutputChannels = activeOutputChannels.getHighestBit() + 1;

        for (auto channel = 0; channel < 1; ++channel)
        {
            if ((! activeOutputChannels[channel]) || maxInputChannels == 0)
            {
                bufferToFill.buffer->clear (channel, bufferToFill.startSample, bufferToFill.numSamples);
            }
            else
            {
                auto actualInputChannel = channel % maxInputChannels; // [1]

                if (! activeInputChannels[channel]) // [2]
                {
                    bufferToFill.buffer->clear (channel, bufferToFill.startSample, bufferToFill.numSamples);
                }
                else // [3]
                {
                    auto* inBuffer = bufferToFill.buffer->getReadPointer (actualInputChannel,
                                                                          bufferToFill.startSample);
                    auto* outBuffer = bufferToFill.buffer->getWritePointer (channel, bufferToFill.startSample);
					auto count = 0; // counter of dropped samples

					scopeDataCollector.process(inBuffer, bufferToFill.numSamples);

					AudioBuffer<float> buffer(bufferToFill.buffer->getArrayOfWritePointers(),
						bufferToFill.buffer->getNumChannels(),
						bufferToFill.startSample,
						bufferToFill.numSamples);

					for (auto sample = 0; sample < bufferToFill.numSamples; ++sample)
					{

						if (inBuffer[sample - 1] == inBuffer[sample] && inBuffer[sample] == 0.0f)
							count++;
					}
				
					//if (count > 1) {
					//	DBG("Dropped samples = " << count << " start sample " << bufferToFill.startSample << ", buffer size: " << bufferToFill.numSamples);
					//	DBG(", xRun: " << deviceManager.getXRunCount() << " cpu time:" << deviceManager.getCpuUsage());
					//}
					for (auto sample = 0; sample < bufferToFill.numSamples; ++sample) 
					{
						outBuffer[sample] = 0.0f;
					}
                }
            }
        }
    }
	//==============================================================================
	void paint(Graphics& g) override
	{
		g.fillAll(juce::Colours::black);
		g.setColour(juce::Colours::white);

		auto area = getLocalBounds();
		auto h = (float)area.getHeight();
		auto w = (float)area.getWidth();

		// Oscilloscope
		auto scopeRect = Rectangle<float>{ float(0), float(0), w, h / 2 };
		plot(sampleData.data(), sampleData.size(), g, scopeRect, float(1), h / 4);
	}

	//==============================================================================
	static void plot(const float* data,
		size_t numSamples,
		Graphics& g,
		juce::Rectangle<float> rect,
		float scaler = float(1),
		float offset = float(0))
	{
		auto w = rect.getWidth();
		auto h = rect.getHeight();
		auto right = rect.getRight();

		auto center = rect.getBottom() - offset;
		auto gain = h * scaler;

		for (size_t i = 1; i < numSamples; ++i)
			g.drawLine({ jmap(float(i - 1), float(0), float(numSamples - 1), float(right - w), float(right)),
						  center - gain * data[i - 1],
						  jmap(float(i), float(0), float(numSamples - 1), float(right - w), float(right)),
						  center - gain * data[i] });
	}

    void releaseResources() override {}

    void resized() override {}

private:
	template <typename SampleType>
	class AudioBufferQueue
	{
	public:
		//==============================================================================
		static constexpr size_t order = 14;
		static constexpr size_t bufferSize = 1U << order;
		static constexpr size_t numBuffers = 5;
		std::atomic<bool> isReady{ false };
		//==============================================================================
		void push(const SampleType* dataToPush, size_t numSamples)
		{
			jassert(numSamples <= bufferSize);

			int start1, size1, start2, size2;
			abstractFifo.prepareToWrite(1, start1, size1, start2, size2);

			jassert(size1 <= 1);
			jassert(size2 == 0);

			if (size1 > 0)
				FloatVectorOperations::copy(buffer[(size_t) start1].data(), dataToPush, (int) jmin(bufferSize, numSamples));

			abstractFifo.finishedWrite(size1);
		}

		//==============================================================================
		void pop(SampleType* outputBuffer)
		{
			int start1, size1, start2, size2;
			abstractFifo.prepareToRead(1, start1, size1, start2, size2);

			jassert(size1 <= 1);
			jassert(size2 == 0);

			if (size1 > 0)
				FloatVectorOperations::copy(outputBuffer, buffer[(size_t) start1].data(), (int) bufferSize);

			abstractFifo.finishedRead(size1);
		}

	private:
		//==============================================================================
		AbstractFifo abstractFifo{ numBuffers };
		std::array<std::array<SampleType, bufferSize>, numBuffers> buffer;

	};

	template <typename SampleType>
	class ScopeDataCollector
	{
	public:
		ScopeDataCollector(AudioBufferQueue<SampleType>& queueToUse)
			: audioBufferQueue(queueToUse)
		{}

		void process(const SampleType* data, size_t numSamples)
		{
			size_t index = 0;

			if (state == State::waitingForTrigger)
			{
				while (index++ < numSamples)
				{
					auto currentSample = *data++;

					if (currentSample >= triggerLevel && prevSample < triggerLevel)
					{
						numCollected = 0;
						state = State::collecting;
						break;
					}

					prevSample = currentSample;
				}
			}

			if (state == State::collecting)
			{
				while (index++ < numSamples)
				{
					buffer[numCollected++] = *data++;

					if (numCollected == buffer.size())
					{
						audioBufferQueue.push(buffer.data(), buffer.size());
						audioBufferQueue.isReady.store(true);
						state = State::waitingForTrigger;
						prevSample = SampleType(100);
						break;
					}
				}
			}
		}
	private:
		//==============================================================================
		AudioBufferQueue<SampleType>& audioBufferQueue;
		std::array<SampleType, AudioBufferQueue<SampleType>::bufferSize> buffer;
		size_t numCollected=0;
		SampleType prevSample = SampleType(100);

		static constexpr auto triggerLevel = SampleType(0.05);

		enum class State { waitingForTrigger, collecting } state{ State::waitingForTrigger };
	};
	class VideoThread: public HighResolutionTimer, public AsyncUpdater
	{
	public:
		VideoThread(
			MainContentComponent& mainComponent,
			AudioBufferQueue<float>& audioBufferQueue,
			std::array<float, AudioBufferQueue<float>::bufferSize>& sampleData)
			: mainComponent(mainComponent)
			, audioBufferQueue(audioBufferQueue)
			, sampleData(sampleData)
		{
			startTimer(40);
		}


		void hiResTimerCallback() override {
			if (isUpdatePending()) {
				cancelPendingUpdate();
			}
			if (audioBufferQueue.isReady.load()) {

				audioBufferQueue.pop(sampleData.data());
				triggerAsyncUpdate();
			}
		}

		void handleAsyncUpdate() override {
			mainComponent.repaint();
		}

	private:
		MainContentComponent& mainComponent;
		AudioBufferQueue<float>& audioBufferQueue;
		std::array<float, AudioBufferQueue<float>::bufferSize>& sampleData;
	};
	//void timerCallback() override
	//{
	//	audioBufferQueue.pop(sampleData.data());

	//	repaint();
	//}
	AudioBufferQueue<float> audioBufferQueue;
	ScopeDataCollector<float> scopeDataCollector{ audioBufferQueue };
	std::array<float, AudioBufferQueue<float>::bufferSize> sampleData;
	std::unique_ptr<VideoThread> videoThread;
    JUCE_DECLARE_NON_COPYABLE_WITH_LEAK_DETECTOR (MainContentComponent)
};
#7

Thanks to the plugin standalone version,
I could try the app with input only (because I don’t know why in the standalone non-plugin app, when I disable output I can’t see the component visualization)

So I can assure that with only input or output the problem does not happen, and it arises when both inputs and outputs are enabled, but I need to make it working this way.
It seems they fight for the audio buffer, any idea what’s going on here?

#8

You don’t take it into account when reading from the buffer though. You read a fixed bufferSize value. So if fewer samples are added than bufferSize you read the wrong size buffer.

I don’t know if this is your problem, but it’s a problem :slight_smile:

#9

change jassert(numSamples <= bufferSize) to jassert(numSamples == bufferSize) and see if it triggers.

#10

It does not trigger, because I have this condition in the collector, inside the if:

while (index++ < bufferToFill.numSamples)
					{
						buffer[numCollected++] = *inBuffer++;

						if (numCollected == buffer.size())
						{
							audioBufferQueue.push(buffer.data(), buffer.size());
							audioBufferQueue.isReady.store(true);
							numCollected = 0;
							break;
						}
					}

And for reading, the atomic flag is taking care of not doing so if the buffer is not full filled

		void hiResTimerCallback() override {
			if (isUpdatePending()) {
				cancelPendingUpdate();
			}
			if (audioBufferQueue.isReady.load()) {

				audioBufferQueue.pop(sampleData.data());
				triggerAsyncUpdate();
			}
		}

But you are right that the assert was misleading, it is corrected now, thanks!

#11

Got ya. Sorry, read the code too quickly this morning :slight_smile:

Does it matter that isReady is never set back to false?

You shouldn’t need the hires timer for this, it just adds another threading complication in.

However I cannot repeat your problem:

#12

Does it matter that isReady is never set back to false?

Oops I’m pretty sure that was there at some point, maybe I left out while doing the tests because it seems to get worse with it (with the whole atomic condition check).

I’m able to reproduce it in different systems, but yes it is hard to catch.

In my case I can reproduce it more or less consistently if the sound generator is already running before starting the app. I can then see the drops in the first seconds while the buffer is filling up.

If you try to start the program a few times and look at this closely you should be able to see it.