Audio buffers library

Development
1

Hi to all,

For a project of mine I’ve started working on some custom audio buffers because I needed booth some normal and circular buffers that share the same API so that I can use them interchangeably in some algorithms without caring of which type they are.
Those buffers should came in two form: a “view” version, that act like a span and permit to operate on a block of memory it doesn’t own, and an “owning” version that allocate and manage his memory internally.
Also I wanted to create an abstraction that permit to iterate over those data and operate on them without using “raw” operation. The library have single channel buffers that can operate on a contiguous memory area and multichannel buffers that operate on multiple single channels. (I’ve not considered an interleaved case)

I somewhat managed to create the first version of the library, with many difficulties, but I’m not fully convinced by some design choices or solutions I’ve found (I’m still learning c++, and I could have made some big mistakes), so I would like to share my work with you hoping that can be useful to someone and to receive some suggestions and help from more experienced developers :slight_smile:

Here’s the link to repository on github: https://github.com/mix359/audioBuffersLibrary

Talking of design choice, one of the first one I’m struggling with is the idea to have in someway the ability to use all the basic functionalities of those buffers interchangeably. My first approach was to write some common interfaces for the various types and using polymorphism with virtual inheritance.
The problem came when I’ve tried to measure the performance of some operation like the iterations over the container. In particular if I’m iterating over a buffer with more than one channels for every iteration I’m creating a temporary “single channel” operation object that can be used to operate on that channel, or iterate over the single samples. Using the virtual inheritance that temporary object must be returned in the way of a pointer and should be allocated and destroyed every time making a huge impact in all the operations (about 100x times slower then doing raw operations using pointers).
So I’ve tried using some new technic offered by variant and concepts that permit to do type erasure on types that are similar without sacrifice too much run-time performance and effectively It’s seams way better in term of performance in my test scenario (it’s only about 2x times slower then the raw operations doing iterations, in -O3 with Apple Clang 18 on my M1 Pro MacBook. Also a strange thing is that boost::variant2 are optimized way better by the compiler than std::variant).
Also not using the polymorphism make some operation that previously had to use common interfaces more simple, for example the iterators of the multi channels buffer previously had to return a unique_ptr that should be de-referenced by the user. Now it returns a wrapper object that expose the same API of the others buffers and internally use the variant.
The drawback it’s that development experience with concepts are not that great and absolutely not comparable to using Interfaces in the code… I haven’t found a way to fully express all the cases and also there’s no currently support from linters/IDE for concepts used as interfaces (no method suggestions/check by the intellisense).
So I’m not fully convinced by this change and because it radically change the API of the library, I want to be sure it’s the right move.

Also I’ve some other doubts that maybe someone can help me on:

  • For the circular buffer I have created a particular channel view that behave differently if you’re doing read or write operation, choosing the right index. It’s the default returned by the circular buffers by getters and iterators. You can also request a channel view that only point to the read or write index. Can this make confusion in the user if for example it use the default view and do read and write operation thinking of operating on the same data?
  • The buffers are currently templated to accept integral or floating pointer types, but I haven’t implemented a way to convert from one type to another. Can be useful?
  • I’ve written the library using only assert for the wrong cases, to not introduce run-time overhead when compiled for production. That make the library in some way not safe for some operation and I’m not fully happy by that.
  • I haven’t find a way to test the asserts I’ve put in the code with catch2 unit tests. Should I use some custom assert macro that raise exception during unit tests so I can test them?
  • How much can be useful to track the clear/dirty state of the buffer like the juce buffers does?
  • How really it’s an improvement to in some way preallocate a bit of space like std::string does so that small buffers doesn’t need to allocate other space in the heap but can live fully on the strack?
  • I haven’t implemented copy/move assignment on owning buffer because I find confusing if it has to only copy/assign the data or also do a resize causing a possible allocation. I prefere that the user do the operation manually so he knows what is he doing. It’s a correct approach?
  • Does make any sense to permit the instantiation of an empty owning buffer and after the resizing/allocation of the memory?
  • I’ve tried to not include getters that return raw pointer to see if it’s possible to avoid them. Also because for circular buffer it’s really hard to return something that it’s correct without the contest. It’s the right choice?
  • I’m thinking of views as temporary object that should only be used to do temporary operation, so I haven’t planned any method to change the memory a view it’s pointing to. Also I’m not in any way protecting the memory those view are pointing to and can lead to operations on a dangling pointer.
  • The single channel buffer returned by getters and iterators of an owning buffer is a view so like for the previous point I cannot guarantee that the memory it’s pointing to remain valid. For example doing some resize operation on the owning buffer in another thread can lead to a reallocation and a danging reference for the view.
  • The buffers doesn’t have any locking mechanism that make them thread safe

Sorry for the long post this is one of my first development of a container in c++ and I would really like to have opinions from experienced developer in the field to understand if I’m on the right path.

Thanks to all
Cheers Mix

2

std::span is quite useful for a non-owning pointer to a buffer, and you can cheaply construct one that points at some ‘owned’ data also.

Then you can write some standalone functions that operate on a std::span. This avoids the duplication of writing two different implementations of each function (one owning, one non-owning) and avoids the overhead of virtual function calls. For extra points, you can even make these functions templated on the sample-type, to avoid even more duplication.
Once you are using std::span, you then also get a bunch of functionality for ‘free’. like std::copy for copying a buffer.

1 Like
3

I’ve been successfully using both span and ring_span (GitHub - Quuxplusone/ring_view: std::ring_span (SG14, P0059R1)) for manipulating non owning “views” over preallocated buffers.

4

Thanks @JeffMcClintock and @kunitoki for the suggestions!
I’m particularly interested in the ring_span and I will dive into the code to see what they’ve done :smiley:
In any case I’ve worked on my version of the buffers that in part already behave like a span, but also have some useful method like the ones on the AudioSampleBuffer here in juce.

My big doubt still remain if can be a good idea to use variant/concepts to use two different objects that behaves similarly but doesn’t loose performance because of polymorphism overhead.
Effectively if I’ve seen whell, the two examples you gave me (span and ring_span) are two different objects that behave similarly but doesn’t have an interface in common.

@kunitoki What do you usually do in a project where you have to use both and have for example a method that should accepts both?

Thanks again

5

Without having looked at your implementation, you can make use of templated base classes and inheritance to create a uniform interface for multiple implementations. We did that for our VCTR library that offers a dynamic vctr::Vector container, a static vctr::Array container and the vctr::Span view class sharing a bunch of member functions. All inherit a common base class template

and then inherit that base class

While all containers implicitly convert to the Span which is a good way of passing various containers to functions defined in other translation units, there are also a bunch of concepts that can used to write function templates that take any of the types like e.g.

template <is::anyVctrWithValueType<float> T>
void foo (T&& arg)
{

}

which is the preferred way when writing header only code since no conversion is performed at all and still it is safe to assume that the type passed in has the given set of member functions. I hope this was not too much off topic, but maybe you’ll find some inspiration in our project for your code :wink:

6

I don’t necessary need to forward one of those in a generic way, after all the usage and access pattern of a ring buffer is different from just being able to randomly access a flat buffer. Usually spans can be constructed from contiguous iterators, so a ring_span can take a span. It really depends on what you are trying to achieve…

7

Many thanks @PluginPenguin,

I’ve studied your code in those days and I can say it’s really interesting :smiley:
Sadly I’ve tried applying the same pattern to my code, but it seam to only partially solve my problems:

If I’ve understood well what you’ve done, you have a base class that it’s a wrapper around some real container stored internally, and expose the various common methods of those containers.
The real container type is passed using a template and constructed/managed by the base class. The implementers of the base class are some sort of alias that prepare the template for the base class.
This create an inheritance with a basic “interface” for all the containers type, but it doesn’t seam to me that can be used as a generic interface to store a generic container on a class.
For example, I haven’t find a way to to store either a vctr::Span or a vctr::Vector in a context like this:

class A {
    vctr::VctrBase<int> m_container;
}

That because I also have to specify the others template parameters of the base class that I don’t know in advance.

Also as you have suggested I have to use a templated parameter restricted by a concept to accept a generic container as a function parameter and I can’t use the base class as a generic interface like before.

I was already playing with concepts and they are an interesting solution with no runtime overhead, but it’s not yet well supported by the linters/intellisense of the IDEs (the intellisense can’t suggest “method” defined using a concept). Among other things I’ve tried declaring some sort of interface using concepts but I haven’t find a way to express some of the more complex methods declaration.

Am I missing something or do you confirm that even with the trick you used of the templated base class there are those limitation and the only way to have some sort of type punning in those situation is to use variant and concepts?

Thanks again
Cheers

8

Thanks for the feedback and the interest.

You are right, using a vctr::VctrBase as a variable directly is not intended to be done. However, for better advice on a pattern I need some more context about your application. Depending on the context the solution for that example class would either be just picking a suitable container type

class A {
    vctr::Vector<int> m_container;
}

or making A a class template like

template <vctr::is::anyVctrWithValueType<int> Container>
class A {
    Container m_container;
}

In most of the situations, the obvious best container choice is clear anyway, so for an example a DSP class like this (just sketched it up for the sake of the example)

/** Computes an FFT on the input samples and returns a Mel mapped band representation. */
class AudioBufferToBandsTransformation
{
public:
    static constexpr size_t numBandsOut = 100;
    using BandArray = vctr::Array<float, numBandsOut>;

    void prepare (size_t expectedBlockSize)
    {
        expectedNumSamplesIn = expectedBlockSize;
        bins.resize (expectedBlockSize / 2 + 1);
        fft.prepare (expectedBlockSize);
    }

    template <vctr::is::anyVctrWithValueType<float> Samples>
    const BandArray& process (const Samples& samplesIn)
    {
        assert (samplesIn.size() == expectedNumSamples);
        fft.forward (samplesIn, bins);
        bandMapping.process (bins, bands);
        return bands;
   }

private:
    size_t expectedNumSamplesIn;
    vctr::Vector<std::complex<float>> bins;
    BandArray bands;
    
    FFT fft;
    MelBandMapping<numBandsOut> bandMapping;
}

I’ts clear what suitable choices for the member containers are. Still that process function can be called with any type of VCTR container if that FFT::forward function maybe had an interface like

void forward (vctr::Span<const float> samples, vctr::Span<std::complex<float>> bins);

if its definition was in a different TU or

template <vctr::is::anyVctrWithValueType<float> Samples,
          vctr::is::anyVctrWithValueType<std::complex<float>> Bins>
void forward (const Samples& samples, Bins& bins)

if it was header only.

So long story short most of the time I know the exact type of member containers but want to be flexible to pass any suitable container types to functions. That’s the ecosystem our library was designed for. Do your use cases differ from what I sketched here?

Being a CLion user I can’t second that concepts are not well supported by the IDE, at least for me CLion instantly gives me auto completion suggestions for member function calls based on the concept when a certain variable is constrained by a concept. For my Windows developer colleagues using Visual Studio with Resharper it’s the same. But maybe intelligence is just not as good as those other tools :man_shrugging:

In general, it takes some time to really get into concepts. I find writing simple concepts and then creating more specific concepts by composing them from those simple concepts the best approach, but again, do you have some more specific real world example? Maybe I can give you a better response then :slight_smile:

9

Thanks again for the reply :slight_smile:

Yes, I’ve a different situation. I’m dealing with something that it’s more similar to a DAW than a plugin, and use a graph with nodes that does single processing. Every node have internally a buffer where it write the output of the processing. Those buffers are usually normal buffer, but there’re particoular nodes that use the circular one (for example to communicate with the soundcard) and the delayed circular one, to introduce a controlled latency.
Those node are connected usign an intermediate structure that keeps the references between the two nodes and also keep a reference to buffer of the previous node in the chain.
That reference is passed to the next node when his processing is called as a parameter of the method.

That’s a really simplified version of what I have:

class NodeInput {
public:
...
    const AudioBufferViewInterface& getSourceNodeOutputBuffer();

protected:
    AudioBufferViewInterface& m_sourceNodeOutputBuffer;
};

class Node {
public:
    void process(const AudioBufferViewInterface& inputBuffer);
...
};

NodeInput input;
Node* node;
node->process(input.getSourceNodeOutputBuffer());

As you can see I have 3 different cases where I need to pass around or store a reference to a buffer that can be of different types, but have some common methods that permit to read the data in the same way.

Initially I’ve developed all the buffers using the virtual inheritance, to have a common interface class to use around like in those cases, but as I have an abstract interface I was forced to move them around as pointers and not references (c++ doesn’t permit to have reference to incomplete types), and to avoid dangling pointers I have started using unique_ptr… but in that way the performance was really bad. (I had to pay for the allocation/deallocation to move around those containers).
And that’s why I’m trying to find another way to achieve the same behavior of an interface that I can use around as a function parameter, return type or a way to store a generic reference.
I’m currently using variants and concepts but they are not as flexible as I need. For example a templated parameter cannot be used in a virtual function, so I can’t use concepts in that contest. (And my Node class use virtual inheritance for the main methods like process).

Do you think the structure in your library can solve those problems in any way?

I’m also a CLion user, and effectively it’s not totally true that the intellisense doesn’t work with concepts… but in my case it works when it want… If you say that it should work, I will try to understand if I’m doing something wrong that it’s blocking the intellisense in any way.

I’m currently writing those types of concepts in a way I’ve found as an examples on some blog posts, but I’m struggling with some particular case where I need to reference a concept as parameter type, and it doesn’t seam to be possible.

For example, you can see here that I’m creating the concepts that represent for example the class AudioBufferView, and I cannot write the concept for the method “copyFrom” because it’s first parameter is a templated parameter restricted by another concept, and I haven’t found a way to use a concept in that situation.
Here’s the concept:

And here’s one of the buffer it should be compatible with the concept “AudioBufferType”:

Any idea on how those methods can be represented as a concept?

Many thanks again for the time you’re spending! I’m really appreciating :smiley:
Cheers