I have been searching for a way to lower the memory used by my plugin (sampler with 3520 FLAC AudioFormatReaders in memory) and I started looking at the implementation of the FLAC reader.
I found a reservoir buffer which on my machine always gets set to 8096 samples (*2 channels) and I’ve asked if this is somehow controllable, here.
There is no answer on that question, yet, but I realised this only amounts to 1/3 of the extra memory I wasn’t sure about (everything else - preloads etc. is accounted for). So I profiled the initialisation of my plugin and discovered that virtually all that extra memory I couldn’t explain before was allocated in:
juce::FlacReader::readSamples(...) FLAC_stream_decoder_process_single juce::FlacNamespace::read_frame_(...) juce::FlacNamespace::allocate_output_(...)
This memory doesn’t get cleaned up after
juce::FlacReader::readSamples() is completed and after a certain point I think it stops growing (probably when each and every reader has read samples from disk).
The problem - with 3520 readers and preload of 4000 samples for each file, I get
~107 MB (preload) + 214 MB (the memory allocated in readSamples) + 214 MB (reservoir buffers) + ~50 MB (FLAC_stream_decoder_new and FLAC_stream_decoder_init_stream) ============================================================= Total: 585 MB
Or otherwise said - the actual preload (which should be the big chunk of data I want to have in memory at all time) clocks at just 18% of the memory used…
Is this how the FLAC reader is supposed to work?
If it does such heavy allocation, I wouldn’t think it will be better creating new readers all the time, right?
And since it has a reservoir of samples, does switching context (file it points to) work safely?