Read sample file into memory in its native format


#1

I need to read entire sample files (typically wav) into RAM for fast random access.  The sample files aren't super-huge (< 4MB), but it's a mobile application and I don't want to use memory unnecessarily.

AudioFormatReader::read() seems to always deliver samples in a 32 bit format (int or float) regardless of number of bits per sample in the file being read.  Since nearly all the sample files I deal with are 16 bits, this seems rather wasteful.

Is the only option to read a block in this format and then convert back down to 16 bit?  Seems like a lot of unnecessary conversion work - am I missing something?


#2

It's probably better to use a MemoryMappedAudioFormatReader rather than actually reading it into memory.

But bear in mind you could also just load the raw file content into a memory block and play it via a MemoryInputStream.


#3

Yep I'm happy to read in the raw file, that's what I've done in non-Juce stuff before.  Trouble is that WavAudioFormat/AudioFormatReader don't have much of an interface for doing this so I wan't sure that was an intended approach.  I might add some gubbins to do this, my version of Juce is pretty hacked around anyway.

In your experience, does memory mapping a file ever cause stutters or blocks when accessed on the audio thread if the OS decides it needs to page in data?  I've often considered trying it, but always thought this is why real time audio programmers decide to do it the hard way!

 


#4

When you malloc a block of memory and copy a file into it, the result is pretty much the same as memory-mapping the file and then touching all the bytes in it.

The OS is allowed to swap-out a block of memory that you malloc'ed just like it's allowed to swap-out parts of a memory-mapped file, so neither is guaranteed to be always-in-memory if memory is tight.


#5

> When you malloc a block of memory and copy a file into it, the result is pretty much the same as memory-mapping the file and then touching all the bytes in it.

Yes I've observed memory usage in realtime performance monitoring tools in the past, and typically you see it increasing during the read (ie. memory touch) rather than at the point of allocation.  Hence my question - in the non-memory mapped case the memory has all been touched and there's a good chance it's going to be instantly available, but when explicitly memory mapped I'm not sure if there will be a stall (and if that stall's worth worrying about) at the point at which it's read by the audio thread.

 

> The OS is allowed to swap-out a block of memory that you malloc'ed just like it's allowed to swap-out parts of a memory-mapped file, so neither is guaranteed to be always-in-memory if memory is tight.

I'm mainly concerned with iOS which doesn't have a backing store for writable pages - the Apple docs on virtual memory say, "Writable data is never removed from memory by the OS. Instead, if the amount of free memory drops below a certain threshold, the system asks the running applications to free up memory voluntarily ... Applications that fail to free up enough memory are terminated."

So it would appear that there's limited benefit to be had from memory mapping on iOS in this case, unless there's a high chance that the loaded samples (or a part of the samples) will never be read.  The only thing I can think of doing would be to respond to the low memory warning by freeing all memory occupied by samples which the app knows haven't been modified in memory, and then immediately memory mapping them again.

Apologies for wandering off topic from the original title, but you reminded me of some questions I've always been meaning to ask!