Storing audio buffers in plugin state

Audio Plugins
1

Hi,

I’m currently storing my plugin’s state like this:

void Processor::getStateInformation (juce::MemoryBlock& destData)
{
    juce::MemoryOutputStream stream(destData, false);
    apvts.state.writeToStream(stream);
}

This is for my sampler plugin, many users have been coming to me asking for a way to save the sample into the saved state. I have dissallowed this for obvious reasons (instead simply saving the path to the sample), but now many people are having trouble collaborating, which I forsaw to a certain extent because that’s gonna happen when you’re sharing file paths.

But now I think its reasonable to allow saving samples to the pugin state, especially for short samples, there are sampler plugins out there that allow this.

My query is how to do this in an elegent way that is backwards compatible.
My first instinct would be to attach an Array Property to the apvts.state, but then I don’t know how effective that would be with audio data. Furthermore I’m not sure what kind of a memory footprint the arrays take up when converting to a memory block:

    juce::MemoryOutputStream stream(destData, false);
    apvts.state.writeToStream(stream);

Any help would be much appreciate.
Cheers

2

Why don’t you simply append the audio buffer to the MemoryOutputStream?

3

that is an option, but then when I decode the memory block, but then in setStateInformation:

void Processor::setStateInformation (const void* data, int sizeInBytes)
{
    juce::ValueTree tree = juce::ValueTree::readFromData(data, sizeInBytes);

I’m not sure how I would figure out that there is a block of audio saved there if it is simply appended to the end of the value tree binary.

4

The binary data that gets saved by the ValueTree actually adds the length of the XML string as the first binary element (actually the second, there is a special magic code that’s inserted before that). So you can skip that many bytes and then read your sample data.

5

I would recommend adding all custom data as ValueTree properties, so basically you need a method to convert AudioBuffers to and from vars. I think there’s an example of the forum, but I can also dig out some code for it…

6

I don’t know how you intend to do that, but you will create considerable overhead when you put binary data into an XML structure.
At best you could base64 encode…
Since the storage the host offers is a binary blob, saving the data similar to what @kerfuffle proposed should be preferred.

7

I personally like how it’s handled in NI Battery. The plugin can handle too many samples to put them all into the state of every single patch you make, but you can save a preset file with the samples incorporated in case you wanna transfer the patch to another computer or sell a preset pack and make it convenient to the users. sometimes you end up opening a project that you have copied from one machine to another and forgot to save the patches like that beforehand. then battery will greet you with an alert window when you open the project on the new machine and there are workflows for selecting a new folder that also contains those samples and it will then automatically search it for them. I’ve rarely had any problems with this solution. Only things like having unique samples on my desktop and then accidently deleting them because I forgot I still use them in some unfinished project, but users nowadays more and more tend to even get around that by incorporating a lot of audio bounces into their workflows

2 Likes
8

Depending on the host, the data may end up encoded inside an XML file anyway. Logic Pro, for example, does that. It’s not very efficient either way but at least if you’re appending the binary data to the MemoryOutputStream directly, it doesn’t get encoded twice.

9

As long as the samples are small, I’d probably also prefer storing it base64 encoded as a property in the ValueTree. The reason is that this allows a lot of flexibility, for instance you can easily store multiple samples in the ValueTree.

3 Likes
10

Interesting, thanks for all your suggestions. I agree that keeping it in the valuetree would be a lot more flexible, but I will have to test to see how big file sizes get.

How would you go about storing it in a value tree, as a base64 encoded string?
How would I go about generating that?

Cheers

11

What’s stopping us from Base-encoding everything and storing it in the value tree? What’s the theoretical size limit, and what are the potential trade-offs?

12

FYI, don’t know if this still the case, but there used to be a limit in chunk size in LogicPro

13

I’ve been storing samples in the value tree as base64 encoded strings. Haven’t had issues with speed, even for fairly long files.

14

very interesting, I’ll have an experiment. What is the ceiling for file size with this encoding?

15

On my 2022 MacBook Air it takes less than 250 ms to load an entire song into memory, encode the file as a base 64 string, add that string to the value tree, fade out the previous sample, and then plot the waveform completely. This still feels pretty snappy for a sample drag-and-drop.

For anything smaller than a song like one shots or loops, the whole process feels basically instant. I didn’t put much effort into making it super optimized, so I bet this can be improved to some degree.

The song on disk is about 55 mb (32 bit stereo 44.1 kHz)
The plugin preset file with the base64 encoded string comes out a little bigger at around 75 mb.
So the base64 string is a little bit bigger but not by any unreasonable amount.

This plugin is meant for shorter files like one shots and loops/fx so this approach works well. If you want to use files that are like 10+ minutes long, it would be better to use another approach. A 10 minute song ends up as 300 mb in the preset file, and an hour long audio file ends up as 1.8 GB.

But for normal one shot and fx samples, the preset file will only be a few megabytes in size and it will feel super snappy for the user.

I’d recommend putting some file size limit. Ableton’s sampler does this too.

3 Likes
16

Thanks for the reply, very insightful.

I’ll have a mess around and do some benchmarks to see what happens!

Cheers

1 Like
17

Update:
I’ve got something semi working, but ive hit a snag using the juce::Base64 functions.

I’m basically using juce::Base64::convertToBase64 and convertFromBase64.

However when I try to encode/decode in the same function, the resulting memory block is a different size to the original data, and my AudioFormatReader fails to read the file.

The right AudioFormat gets picked so something is going right, but I can only assume that something has been corrupted in the base64 translation steps.

How have you gone about convert to and from whilst maintaining data integrity?

Cheers

18

Can you share the code?

How I would debug this:

  • try encoding a very small file
  • print out the base64-encoded string
  • use another base64 tool to also encode the same file (plenty of websites do this)
  • compare the two outputs to verify whether the base64 encoding worked OK
1 Like
19

A caveat is, that some writing functions are only called once the writer is destroyed. That is true for the AudioFormatWriters as well as the MemoryOutputStream that writes into the memory block.

You need to add clever scoping to make sure the writing to the MemoryBlock is finished.

Another cayeat: juce::Base64 and juce::MemoryBlock.toBase64Encoding are not compatible. The one in MemoryBlock is also not compatible with standard base64 en-/decoders.

2 Likes
20

Are you trying to serialize the audio file itself or a juce buffer? In my case I’m just serializing the buffers array of floats as base64 and storing some other important information like number of channels, number of samples, and sample rate, and file path in the value tree.