Change to AudioSampleBuffer to handle out-of-memory?


#1

Hello!

For my application I keep one large sample buffer in memory.

In testing, I’ve been loading large files (2 hours) - If I try to resize my AudioSampleBuffer to handle it, I get a SEGV.

I’d like to report to the user that I can’t open the file, and not crash!

Now, it seems to me that, for my purposes anyway, this can be done with a tiny change AudioSampleBuffer::setDataToReferTo: void setDataToReferTo (float** dataToReferTo, int numChannels, int numSamples, bool freeExistingSamples = true) noexcept;

This lets me take the existing samples, realloc them, check for failure, and on success replace them in the AudioSampleBuffer, and it’s completely code-compatible with existing code.

EDIT: uh-uh, that quite doesn’t do it, because it still copies the data unnecessarily… but there are a lot of fairly simple ways to deal with this, I welcome comments!


#2

I’m intrigued as to why you’re loading the whole thing into memory… To devote gigabytes of memory to cache a 2hr file, and to incur the initial loading time that must be required to read it all, you must really need to access any point in the file with super-low latency… but I can’t think of a situation where it’d be more efficient to do that than to just have a massive circular buffer getting filled by a background thread?


#3

I am currently struggling with a similar issue:

We have a bug report in from a user that wants to load huge sample libraries, but still uses a 32 bit host, so he hits the 4gb limit pretty easily and the plugin crashes within AudioSampleBuffer::allocate() .
I’d leave to keep on using the AudioSampleBuffer to do the memory allocation for me (instead of using ::setDataToReferTo) - but I really need a way to handle the out-of-memory case, but I’d like to avoid replacing AudioSampleBuffer with a modified copy just for this issue.

Maybe setSize() could be changed so that it returns true if the allocation was succesfull? Or am I missing that it’s somehow already possible to properly handle out-of-memory conditions right now?


#4

Hmm. I guess I should probably make sure it throws a std::bad_alloc when if fails to re-allocate.


#5

That’s also fine, I’m not a big fan of using the return value either. Anything that will allow us to not crash the plugin (& host!) is very welcome.


#6

Ok, I’ll sort that out soon. I was planning on giving the HeapBlock a template parameter to tell it whether to throw or not when it fails, so that’d do the trick in this case.


#7

Much of the time the audio comes right from a CD, and we don’t want to keep spinning the CD when people are using the program, so I buffer everything in memory. I did think of buffering from disc to disk and then back into memory but that was even more work.

It works really well - the silence when the CD spins down is palpable (and on a laptop, the disk spins down even if you are reading a disk file).

Now, the program can even fit a whole CD in memory at one time on any modern machine, but now I’m testing Really Large Files from disk - which share the same codepath.

Regardless, I’d like to catch out of memory errors. Someone might simply have a tiny amount of memory left, or open a vast number of windows, or something I haven’t thought of.

This isn’t urgent by any means, but by the time I finish my program I would like it to be that there is no way, contrived or not, to make it crash.


#8

Ah, I see… I hadn’t thought about it being form a CD. Will get it sorted out soon!