[DSP module discussion] New AudioBlock class

General JUCE discussion
1

Today, if you want to store/read/process an array of samples (say float), you have a lot of possibilities to do so in JUCE :

  • Array
  • HeapBlock
  • some InputStream / OutputStream classes
  • AudioData::Pointer
  • AudioBuffer
  • AudioSampleBuffer
  • AudioSourceChannelInfo
  • and now the AudioBlock class !

And we shouldn’t forget the float[] or std::vector / std::array classes as well… Confusing isn’t it ?

So what do we do if we want to manipulate audio samples in JUCE for general purposes tasks ? For example in a plug-in, where we need to do stuff like processing incoming samples, storing some samples for additional processing ?

In general, I tended to use mostly the AudioBuffer class (or the AudioSampleBuffer which is just the float templated version of AudioBuffer). Its API / interface makes clear that it is a better candidate for audio samples than the Array class, and something I discovered recently, you get the alignement of your data for SIMD processing as a bonus, making it useful for storing state variables or filter coefficients for example as well.

But when I made a new processing class, I generally needed to provide a process function somewhere. That’s where I got some headaches before the release of the DSP module. What should I provide there as argument to get a reference to my audio samples ? An array of floats ? The number of samples ? Multiple arrays of floats ? An array of array of floats ? How do we take multi channel processing into account ? How do we process multiple sub blocks of the main audio buffer ?

These questions were some of the fundamental ones the JUCE team asked itself during the development of the DSP module. And the new class AudioBlock was one of the answers, mainly the work of Fabian and Jules.

Now, thanks to the AudioBlock class, it is possible to provide one single object as an argument in process functions, with a reference to audio data without any extra memory allocation, providing a number of channels information + an index for the sample we want to process first. The AudioBuffer class can still be used for storing the actual data. Then some AudioBlocks can be created, made from the buffer and some subBlock functions locally. It’s also very useful if you want to process your data in small chunks whatever the size of the input audio buffer.

That’s why most of the new classes in the DSP module use extensively AudioBlocks, and that’s what the JUCE team is going to do mostly in the future developements I guess. A lot of things were easier for me to code (in the Oversampling class for example) thanks to the AudioBlock class.

What do you think of this new class ?

7 Likes
2

If I understand correctly the main difference between AudioBlock and AudioBuffer is that you can pass AudioBlock by value and AudioBuffer by reference. Otherwise, AudioBlock provides some methods for simple math and might be cleaner as AudioBuffer also contains logic to hold its owned sample data.

If I didn’t missed anything and these are the only differences I probably wouldn’t have created a new class as it duplicates a lot of existing behavior. On the other hand, I didn’t keep up with some other late design decisions (e.g. replacing const something with auto, losing constness) so maybe I’m just becoming a bitter old programmer. :slight_smile:

3 Likes
3

Hello !

That’s one of the differences. Another one is that you can make an AudioBlock from the channels 2 and 3 of an AudioBuffer with the samples from 45 to 78. That’s what makes it specifically interesting :wink:

4

After trying the new Oversampling class, using AudioBlock indeed makes sense as it can be easily returned from a method (as used in Oversampling::processSamplesUp()).

5

…other than just:

    AudioBuffer<float> buffer (1024, 2);
    int start = 45;
    int num   = 78;
    float* ptr[2] = { buffer.getWritePointer(2, start),
                      buffer.getWritePointer (3, start) };
    AudioBuffer<float> subBuffer (ptr, 2, num);

(…and I think you could write that in one line…)

Personally I’m missing the point of a second class.

So if I understand that correctly, that simply means, that there is a copy constructor, that doesn’t copy the data, but just refers to it in the copy?

IMHO the risk of diverging features makes me think it is a bad idea and defeats DRY. But that’s just my opinion… I feel like ckhf here (especially the second paragraph :wink: )

6

I let @fabian and @jules tell you their opinion on this thing, but now my last argument in favor of AudioBlock is that it’s easy when I design a new processing class to take an AudioBlock as an argument, or to return an AudioBlock at the end of the function. Any other design makes things more complicated for the API or the number of functions.

7

From how I understand AudioBuffer, it could also be used to be returned from a function similar to AudioBlock. But with AudioBuffer it isn’t directly clear whether the returned object owns the sample data or simply holds a reference.

8

I think you’re undervaluing the fact you can pass by value, this view of memory into a processing function and not have to worry about object lifetime! It makes more a much nicer API overall.

9

The motivation for the AudioBlock comes from the fact that we initially wanted a bunch of very simple FloatVectorOp-like methods in the dsp module which essentially work on just a pointer and a size.

We also wanted to add support for multiple channels though. Instead of now adding a bunch of parameters to every function like numSamples, numChannels, channelPtr we decided to write a very light-weight AudioBlock class which is essentially just a wrapper of the above parameters.

It’s good that AudioBlock does not own any of the memory. This way, you can guarantee that most methods in the dsp module are lock-free - they will never allocate memory. AudioBuffer, for example, would allocate memory in certain cases even when doing simple things like creating a sub-buffer - if the channel count is high enough.

Passing by value objects does not only make returning buffers easier via return-values (as was mentioned above). But it also gives the optimiser much better chance at optimising your function [1] as it knows that the AudioBlock cannot be changed outside of the function being called. Obviously, there is a trade-off that the compiler must copy the value, but on a 64-bit machine, AudioBlock will take up 20 bytes of memory - small enough that you can be pretty sure that it will all be passed inside registers. As a comparison, the AudioBuffer class is not very efficient: it takes up 288 bytes - that would be 72 samples - so AudioBuffer has roughly as much memory overhead as common buffer sizes.

[1] https://www.youtube.com/watch?v=eR34r7HOU14&feature=youtu.be

5 Likes
10

Hi @fabian,

thank you very much to put some insights and numbers to the figure.
I get the idea, and I think it is very good to separate the memory allocation/ownership from the manipulation class. And I support to simplify that, so that was a very good thing, to start that development.

What bothers me though is to have two classes for the same purpose. When I lay out my project, I will have to decide, which one to choose, and sure as hell I will regret it further down the line and have to refactor stuff, because the other one proves to be better for what ever reason.

IMHO it would have been better, to separate the memory management from the AudioBuffer to aggregation rather than built in functionality, and stick with the AudioBuffer being eventually more lightweight after that changes.
Because

  • either the AudioBlock will lack functionality and people will have to decide which one to choose,
  • or the AudioBlock gets additional functionality, ending up as big and inflexible like the AudioBuffer, and everything is like before, except that you have two diverging development threads.

From a users perspective it is very bad to have to choose between two different classes for the same purpose. If you can merge these two classes, then please do! One class for one job should be the goal.

Cheers!

4 Likes
11

I think this is why I’m going to stick with my architecture. Makes far more sense than this class and doesn’t solve the memory management problem anyway…

12

Oh, how I wish we could do something like that. But it’s not possible without a major breaking change. This would probably break every single JUCE app/plug-in out there which is something we are not prepared to do.

AudioBlock may get more functionality overtime but - we defined it as a lightweight, lock-free [1] class - and thus, by definition, will never, ever do things like memory management or channel pointer management. That’s the job of AudioBuffer. AudioBlock will never become as big and inflexible like AudioBuffer because AudioBlock is a different tool in the toolbox.

I don’t think they have the same purpose. Before we did the DSP module we asked a lot of JUCE users how they would like the interface with the DSP module. The vast majority said: “Just give me lots of DSP functions with a simple pointer and number of samples.” Which makes sense because you can apply these algorithms in so many different ways - in scratch buffer, in a sliding window algorithm, etc.

This is what the AudioBlock class is about - it’s a pointer to channel pointers along with numberOfChannels and numberOfSamples. Nothing more. Think of AudioBlock like the HeapBlock class but for audio.

In addition AudioBlock does not need to be only float or double, it can also use SIMDRegister<float> and SIMDRegister<double> - something that wouldn’t really make sense for AudioBuffer as it directly interfaces with the DAW’s and audio device’s buffers.

[1] Except for one method with is the HeapBlock allocation method.

13

I feel the need to repeat my analogy from earlier, because nobody seemed to understand it, and it does really sum this up very well.

These things are all the same concept:

AudioBuffer -> AudioBlock 
String -> StringRef 
std::string -> std::string_view
std::vector -> gsl::span

You have heavyweight, allocating containers. And you have lightweight, non-allocating, wrappers for passing around EITHER references to the contents (or part of the contents) of the heavyweight containers, OR data from other sources. These are very very different use-cases and both are important - I think that anyone who’s annoyed by our use of two different classes may not properly understand the reasons for it.

Ideally I’d have figured all this stuff out properly back in 2003 and the two classes would look the same except for their constructors and a few mutating methods. We’re doing our best to retro-fit an elegant solution with minimum breakage to existing code, but I don’t really think it’s too bad!

2 Likes
14

I had a thought of that over the weekend, I actually wanted to give it a go in a PR, but it will probably take a week, before I find the time to do that:

What if the AudioBuffer<SampleType> simply inherits AudioBlock<SampleType>. You can then implement the existing AudioBuffer manipulation methods (copyFrom etc) to call the correspondent methods in AudioBlock and deprecate them for JUCE-6. So the heavyweight class has the same interface as the lightweight, same as String and StringRef?

Just a thought

15

just throwing it out there, but what is so bad about breaking existing code if the final solution is 100x better than the old code? Apple releases a new phone every year and changes iOS ALLLLL the freakin’ time, and the users adapt just fine. Same for Facebook and Youtube and Gmail. People get over those changes in their familiars pretty quickly. just sayin’… There’s a reason no one develops 32-bit plugins or TDM plugins…

1 Like
16

We do breaking changes all the time, but there’s a balance between how much hassle it causes people and how much visible improvement there is. For a very common class like AudioBuffer where people will use it in hundreds of places in their code, we need to be very very cautious about changes because it could trigger a lot of rewriting.

17

I say you take 2 weeks, fork the repo and design the API the way you think it should be designed, now that you have 15 years of hindsight and small changes. Maybe you’ll end up with an API change like how Apple ended up changing everything over from ObjC to Swift to make programming easier for the developers.