Hi to all,
For a project of mine I’ve started working on some custom audio buffers because I needed booth some normal and circular buffers that share the same API so that I can use them interchangeably in some algorithms without caring of which type they are.
Those buffers should came in two form: a “view” version, that act like a span and permit to operate on a block of memory it doesn’t own, and an “owning” version that allocate and manage his memory internally.
Also I wanted to create an abstraction that permit to iterate over those data and operate on them without using “raw” operation. The library have single channel buffers that can operate on a contiguous memory area and multichannel buffers that operate on multiple single channels. (I’ve not considered an interleaved case)
I somewhat managed to create the first version of the library, with many difficulties, but I’m not fully convinced by some design choices or solutions I’ve found (I’m still learning c++, and I could have made some big mistakes), so I would like to share my work with you hoping that can be useful to someone and to receive some suggestions and help from more experienced developers
Here’s the link to repository on github: https://github.com/mix359/audioBuffersLibrary
Talking of design choice, one of the first one I’m struggling with is the idea to have in someway the ability to use all the basic functionalities of those buffers interchangeably. My first approach was to write some common interfaces for the various types and using polymorphism with virtual inheritance.
The problem came when I’ve tried to measure the performance of some operation like the iterations over the container. In particular if I’m iterating over a buffer with more than one channels for every iteration I’m creating a temporary “single channel” operation object that can be used to operate on that channel, or iterate over the single samples. Using the virtual inheritance that temporary object must be returned in the way of a pointer and should be allocated and destroyed every time making a huge impact in all the operations (about 100x times slower then doing raw operations using pointers).
So I’ve tried using some new technic offered by variant and concepts that permit to do type erasure on types that are similar without sacrifice too much run-time performance and effectively It’s seams way better in term of performance in my test scenario (it’s only about 2x times slower then the raw operations doing iterations, in -O3 with Apple Clang 18 on my M1 Pro MacBook. Also a strange thing is that boost::variant2 are optimized way better by the compiler than std::variant).
Also not using the polymorphism make some operation that previously had to use common interfaces more simple, for example the iterators of the multi channels buffer previously had to return a unique_ptr that should be de-referenced by the user. Now it returns a wrapper object that expose the same API of the others buffers and internally use the variant.
The drawback it’s that development experience with concepts are not that great and absolutely not comparable to using Interfaces in the code… I haven’t found a way to fully express all the cases and also there’s no currently support from linters/IDE for concepts used as interfaces (no method suggestions/check by the intellisense).
So I’m not fully convinced by this change and because it radically change the API of the library, I want to be sure it’s the right move.
Also I’ve some other doubts that maybe someone can help me on:
- For the circular buffer I have created a particular channel view that behave differently if you’re doing read or write operation, choosing the right index. It’s the default returned by the circular buffers by getters and iterators. You can also request a channel view that only point to the read or write index. Can this make confusion in the user if for example it use the default view and do read and write operation thinking of operating on the same data?
- The buffers are currently templated to accept integral or floating pointer types, but I haven’t implemented a way to convert from one type to another. Can be useful?
- I’ve written the library using only assert for the wrong cases, to not introduce run-time overhead when compiled for production. That make the library in some way not safe for some operation and I’m not fully happy by that.
- I haven’t find a way to test the asserts I’ve put in the code with catch2 unit tests. Should I use some custom assert macro that raise exception during unit tests so I can test them?
- How much can be useful to track the clear/dirty state of the buffer like the juce buffers does?
- How really it’s an improvement to in some way preallocate a bit of space like std::string does so that small buffers doesn’t need to allocate other space in the heap but can live fully on the strack?
- I haven’t implemented copy/move assignment on owning buffer because I find confusing if it has to only copy/assign the data or also do a resize causing a possible allocation. I prefere that the user do the operation manually so he knows what is he doing. It’s a correct approach?
- Does make any sense to permit the instantiation of an empty owning buffer and after the resizing/allocation of the memory?
- I’ve tried to not include getters that return raw pointer to see if it’s possible to avoid them. Also because for circular buffer it’s really hard to return something that it’s correct without the contest. It’s the right choice?
- I’m thinking of views as temporary object that should only be used to do temporary operation, so I haven’t planned any method to change the memory a view it’s pointing to. Also I’m not in any way protecting the memory those view are pointing to and can lead to operations on a dangling pointer.
- The single channel buffer returned by getters and iterators of an owning buffer is a view so like for the previous point I cannot guarantee that the memory it’s pointing to remain valid. For example doing some resize operation on the owning buffer in another thread can lead to a reallocation and a danging reference for the view.
- The buffers doesn’t have any locking mechanism that make them thread safe
Sorry for the long post this is one of my first development of a container in c++ and I would really like to have opinions from experienced developer in the field to understand if I’m on the right path.
Thanks to all