Spectrogram Plugin

dsp_module

#1

I am making a plugin that displays a spectrogram of the audio in real time. I have a version that seems to be working well with Audacity. I am posting this to share my approach and get your comments.

I used the JUCE FFT tutorial as a starting point, even copying some of the code. One fundamental consideration was how to divide the work between the PI processor and the PI editor. I wound up doing most of the work, up to and including creating the spectrogram image, in the processor, mainly in the processBlock method. I pass a reference to the image to the editor in the create editor function and the editor uses a timer callback to display the current image.

Some other differences:
I calculate the FFT using both the current and previous audio blocks. So I have 2x sized blocks with 50% overlap.
I apply a Hann window before calculating the FFT.
I convert the FFT magnitude to dB.
I determine the FFT level for a given pixel using interpolation. This makes a big difference at the low end of the logarithmic scale, where integer mapping from pixel index to FFT index gets very coarse.

As stated, it seems to work quite well with Audacity (Win32). Unfortunately, unlike my previous two plugins, it crashes Ableton on start-up. Not sure how to fix this.

Comments appreciated!


#2

Have you debugged the crash? If not, debug the crash :wink:


#3

Not sure about your issue with running in Ableton, but I know that you 100% don’t want to be doing any graphics stuff in the processor - especially not in processBlock!

I use a timer in my editors to repeatedly get an array of the past N samples from the processor and then process the FFT, apply windowing, etc. etc. in the editor. The processor just pushes samples to a FIFO and waits for the editor to ask for the samples.


#4

Thanks for the quick feedback - this is just the kind of advice I was looking for.

I appreciate that the PI processor is subject to hard RT constraints and should process blocks expeditiously. My reasoning for passing images rather than audio blocks is that I am not too concerned about missing an image or displaying the same image more than once. That is, I do not need to synchronize the editor and the processor that precisely. For the audio block option, I don’t know the best way for the processor to alert the editor that a block is ready. I’ll think about this a bit, but would appreciate suggestions.

Thanks again!


#5

If you’re updating the image in your processor block then, depending on the block size, you may be needing to update hundreds of times a second - which is way above what most monitors can display (60Hz).

You could just have a getFFTBlockIsReady() method in your processor which your editor can call before then calling a getFFTBlock() method if the block is ready.


#6

I took Im_Jimmi’s advice - partially. I still do the FFT calculation in the processor, since I regard this as signal processing. But I moved the image processing, including interpolating FFT values, to the editor. So my getFFTblock() gets FFT values (dB magnitude) rather than sample values. The good news is that it still works with Audacity. The better news is that it now works with Ableton!

By the way, since samplesPerBlock is not available until prepareToPlay() is called, I do some dynamic allocation using scoped pointers. I hope this isn’t a no-no.

Thanks again to the gang for your help.


#7

It is a no no. prepareToPlay gives you the max number of samples you will get. Use it for your allocations.
I would still move all the FFT process outside the main loop, so that it only copies the new data in buffers and then you swap buffers.
If you can live with dropped images, you can live with this workflow and then the plugin won’t take as much CPU usage.


#8

I am doing my allocation in prepareToPlay(). I do have one concern, however. Smart pointers weren’t around when I got my copy of Kernighan and Ritchie, and they are a bit new to me. I assume that each time I assign a new object to a ScopedPointer, any previous object is automatically deleted. Is this true? I hope so since I imagine prepareToPlay() may be called more than once.

I will consider moving the FFT calculation to the editor - it should be easy enough to do.


#9

You are in C++, not C. K&R is not relevant here.


#10

IMHO…you are thinking about this from the wrong angle. Its not that you can’t re-use a ScopedPointer, you can. But, re-using a ScopedPointer defeats the purpose. They are intended to perform a task, and then go away automatically when out of scope.

So, perhaps you need to limit your scope and have one-use ScopedPointers with names that clearly define their purpose, and then go away. That will surely result in more reliable code, since it is easy to lose track of ownership if you pass pointers in and out of a single ScopedPointer.

…not to mention that ScopedPointers, as much as everybody loves them, are effectively deprecated. So, we all need to be switched over to std::unique_ptr (with std::make_unique).


#11

Yes, that is correct. Same is true for std::unique_ptr.
And ScopedPointer is not deprecated, just the JUCE team stops using them, and encourages people to move to unique_ptr as well. But it is not a must.
This is just to avoid code duplication, now that the >C++14 versions implement the same stuff, that juce’s smart pointers did.


#12

I did, in fact, say “effectively deprecated” :wink:.

But, as you said, with C++14 being preferred, ScopedPointer became somewhat redundant (although we love its simple usage model!). But more than that, std::unique_ptr does some things a bit better/safer, so it is good to be in the habit of using it anyway.


#13

Thanks again to all for your help. I have migrated all of the heavy calculation to the timerCallback function in the editor, which uses a processor method to get the most recent (dual) block of samples and then calculates and plots the STFT. Also, most of the dynamic allocation occurs within the scope of this callback. Finally, I replaced most of the scoped pointers with std::unique_ptr. I could not figure out how to pass a std::unique_ptr to an array to the getSampleBlock() method, so I stuck with scoped. The plugin still works!