Convolution in a Console App

General JUCE discussion
1

I’m trying to make a simple console application that takes in two files (say source and impulse) and convolves one with the other using the Convolution class in JUCE. I am able to load both the source and the impulse and play the source back, but I am unsure of how/where to actually call process() on the convolution itself with the source as input.

I’m very good with nitty-gritty, sample-level DSP and know my way around C++ but the JUCE framework is confusing me. Below is my class:

class MyConvolver: public Convolution 
{
public:

    // members
    ProcessSpec spec;
    ScopedPointer<AudioFormatReaderSource> source;

    MyConvolver(File &sourceFile, File &hrir, double sampleRate, int samplesPerBlock) {
        spec.sampleRate = sampleRate;
        spec.maximumBlockSize = samplesPerBlock;
        spec.numChannels = 2;
        source = new AudioFormatReaderSource(formatManager.createReaderFor(sourceFile), true);
        this->prepareToPlay(impulse);
    }

    void prepareToPlay(File &impulse)
    {
        // spec.numChannels = getTotalNumOutputChannels();
        this->reset();
        this->prepare(spec);
        this->loadImpulseResponse(impulse, Convolution::Stereo::yes, Convolution::Trim::yes, 0, Convolution::Normalise::yes);
    }

    void playSource(juce::AudioDeviceManager &audioDeviceManager) {
        AudioIODevice* device = audioDeviceManager.getCurrentAudioDevice();

        if (device && source) {
            std::cout << "yay" << std::endl;
            source->prepareToPlay (device->getDefaultBufferSize(),
                                   device->getCurrentSampleRate());
            ScopedPointer<AudioSourcePlayer> player = new AudioSourcePlayer ();
            player->setSource (source);
            audioDeviceManager.addAudioCallback(player);

            // actually makes the sound???
            while (source->getNextReadPosition() < source->getTotalLength())
                Thread::sleep(100);
        }
    }
};

Perhaps subclassing Convoltion is unnecessary here?

(I think my larger difficulty is with understanding signal flow in JUCE in general. Where exactly is audio written? When is audioDeviceIOCallback() called versus getNextAudioBlock()? I think I understand that JUCE needs to have an audio callback to a device manager to actually process audio, but I’m not sure how to tell it to use the process() method or send the ProcessContext to it.)

2

The way your code is currently written, doesn’t allow using for example the DSP convolution processor. You’d need to write either your own AudioIODeviceCallback or an AudioSource subclass that processes the convolution. Subclassing from the DSP Convolution class isn’t going to achieve anything useful here, and you probably shouldn’t do it.

3

Glad I caught your edit, I started chasing down AudioProcessor! So AudioSource has a getNextAudioBlock() method; I can expect this to be called each block then? And I suppose that I would add the AudioSource with audioDeviceManager.addAudioCallback(myAudioSource)?

The demo has the convolution process itself inside a struct so that also threw me off.

4

If you make an AudioSource subclass, you’d still need the AudioSourcePlayer object. (Which adapts an AudioSource to be an AudioIODeviceCallback used by the AudioDeviceManager.)

1 Like
5

Awesome, thanks for your help. I’m sure I’ll be back in a few hours.

6

I made a quick and simple implementation : (not guaranteed to be 100% correct, but it does play the sound file via the convolution processing)

1 Like
7

Awesome, this works well (I modified it a bit) except that the sound starts cutting out (only small blocks seem to get through) after about 10-15 seconds. No errors are thrown and my impulse is much shorter than that so I’m not sure why it seems to stop.

Is there some sort of allocation going on under the hood that I’m unaware of? Why might this be occurring?

EDIT: I discovered that the getNextAudioBlock() method goes from being block/samplerate seconds (512/44100 for me) to every 1/4 second or so after about 900 blocks. Any ideas? I’m on Linux if that matters…

EDIT TWO: It’s definitely in the Convolution class. process() commented out it just spits the input to the output as expected.

8

Ok, if the size of the incoming buffer is changing (which I suspected would be one of the problems that could be in the code), the code in the getNextAudioBlock needs to be changed to take that into account.

Something like this works here but I think I am not actually getting the changing buffer sizes on Mac Os :

void getNextAudioBlock(const juce::AudioSourceChannelInfo &bufferToFill) override
    {
        if (!m_readersource)
            return;
        m_readersource->getNextAudioBlock(bufferToFill);
        juce::dsp::AudioBlock<float> block(bufferToFill.buffer->getArrayOfWritePointers(),
                                        bufferToFill.buffer->getNumChannels(),
                                           bufferToFill.startSample,
                                           bufferToFill.numSamples);
        m_convolution.process(juce::dsp::ProcessContextReplacing<float>(block));
        bufferToFill.buffer->applyGain(0.01f);
    }
9

So the size of the buffer passed to getNextAudioBlock() is not guaranteed to be the same size each call? What’s the reasoning behind this?

Also, that wasn’t it. The buffer is the same size each call (examined via bufferToFill.numSamples). I also tried increasing the latency of the Convolution but it didn’t seem to change it when I checked via .getLatency(); I still received 0.

10

The buffer itself is probably the same size for each call, but the section of the buffer you are supposed to process can vary. That’s why the code implementing the AudioSource::getNextAudioBlock should take into account the startSample and numSamples of the AudioSourceChannelInfo object and not just process the whole AudioBuffer contained in it. This was a mistake in my original code, even if fixing it didn’t solve your particular problem.

Is the IR you are using “long”? The Juce convolution uses more CPU as the IR length increases. Did you test using a release build of the code? What changes did you do the code compared to code I posted? Are you letting the audio run past the end of the audio file to get a reverb tail happening? Maybe that requires some special handling where you pass silence into the convolution instead of the audio you get from the AudioFormatReaderSource. (I think the AudioFormatReaderSource should itself produce silence when it has read to the end of the file, but I am not 100% sure at the moment.)

11

I’m not sure I follow:

The buffer itself is probably the same size for each call, but the section of the buffer you are supposed to process can vary.

When you say “buffer”, what exactly do you mean? Is it the block of audio that is to-be-processed before sending to the output (i.e. actually writing samples to the hardware out) or the section of the IR that needs to be convolved at “this” time? I’d thought that the sound processing (in general, not just in JUCE) always gives a block of, say, 128 samples (or whatever is set on the audio device), then passes them as a buffer to a function that is of one’s own design. I was reading that apparently some hosts for plugins can give different buffer sizes and one needs to be ready! Very different than from what I’m used to (writing plugins for SuperCollider and Pd).

The IR is not very long, maybe 0.5 seconds; input file much longer. Tested with release build, same issue. The main changes were organizational and most of the processing is left alone (since it isn’t very complicated, I just didn’t know where to put it).

BUT! I realize that I’m maxing out the CPU even with one Convolution. Bypassing either by commenting out process() or by setting bypass to true in the ProcessContextReplacing makes the glitching stop and the audio pass through as expected. I’ve tried to add latency (and change the head size) to the Convolution in the constructor but it doesn’t seem to persist. That is, if I call .getLatency() right after constructing, it gives what I expect but if I call it later in the prepareToPlay() or getNextAudioBlock() it returns to 0. I try setting like this:

const dsp::Convolution::Latency lat{ static_cast<int> (4096) }; // get a Latency struct
dsp::Convolution myConv(lat); // make a Convolution with the Latency

// just normal stuff
juce::dsp::ProcessSpec spec;
spec.maximumBlockSize = 512;
spec.numChannels = 2;
spec.sampleRate = 44100;
myConv.prepare(spec);
myConv.loadImpulseResponse("myIR.wav", juce::dsp::Convolution::Stereo::yes, juce::dsp::Convolution::Trim::yes, 0, juce::dsp::Convolution::Normalise::yes);
DBG(myConv.getLatency()); // prints 4096

Then if I call in prepareToPlay() or getNextAudioBlock():

DBG(myConv.getLatency()); // prints 0

Any thoughts? I could totally be creating wrong but I looked at the source and it seems to be right… lat does not get deallocated (checked via dgb).

Also, thanks so much. You’ve been helpful here and in at least a dozen other threads I’ve read.

12

I haven’t so far used any of the additional options like latency in the Convolution class, so I don’t know how those could be expected to work. I suspect the latency thing might be a red herring.

I by the way still left in a possible error in the getNextAudioBlock code : the final applyGain call was added there to reduce the volume of the output because it was too loud on my end with the IR I was using. But the wrong overload of the applyGain is used, since it would affect the whole buffer, which is not necessarily correct when using AudioSources and AudioSourceChannelInfos. Instead this overload of applyGain should be used :

Probably, this won’t fix the issues on your end either, but I just pointed this out just in case.

I don’t really have other ideas what you could try. The audio glitches you are getting could be because of CPU overload or because of writing into the wrong section of the buffer or some other issue. You would need to show the whole code you are using, so it could be tested.

13

There’s an implementation difference in Juce for AudioProcessors and AudioSources :

In AudioProcessors (used mostly for plugins) you get passed an AudioBuffer directly and the size of that can potentially vary between each processBlock call. There’s a mechanism that allows doing that without memory allocations or deallocations happening. In AudioProcessors, the samples always start at index 0 of the AudioBuffer.

In AudioSources the section or region you have to process can potentially vary between each getNextAudioBlock call. That is determined by the startSample and numSamples members of the AudioSourceChannelInfo. You should not work based on the size ( getNumSamples()) of the AudioBuffer contained in the AudioSourceChannelInfo.

Whether these varying buffer sizes/sections actually happen or not, will wildly vary between platforms, audio devices, plugin formats and host applications. Your code should in any case be prepared to deal with this.

1 Like
14

False alarm (I’m an idiot). I had an argument which read a file that indicated both sources and IRs. Turns out I forgot to edit the IR file and the paths pointed to the sources… So goes programming! (and many, many hours, haha). But the latency thing is still a mystery for now.

And thanks for the info on the differences between AudioProcessors and AudioSources. Will be extremely helpful.

1 Like
15

It would be useful to get into the bottom of the getLatency thing, it might be a bug in the Juce code or some kind of issue in the documentation. (I didn’t test it yet myself, but I trust you did get those inconsistent results you reported.)

16

Indeed. I’ll make a simple mock up to try and suss it out. Will advise.

17

So I figure this one out. I was trying to pass in the latency with the constructor but it doesn’t seem possible. Has to be set when defining the member variable:

private:
    juce::dsp::Convolution m_convolution {juce::dsp::Convolution::Latency{1024}};