External Data Provider issues and limitations

  1. Apparently at the moment default values for arrays are not supported by annotations, which makes it impossible to default initialize external array if the host doesn’t supply the value.

    image
    Float_Array throws Unsupported property value data type error.

    I’m trying to hook up my ExternalDataProvider to provide sample data and let the compiler/linker sort out default values for everything else, like these float values. But apparently it doesn’t like default value for array, so there’s no way to initialize external array value if you’re using your own ExternalDataProvider? :thinking:

  1. External values expected to be read from files.

    I have a simple VirtualFileValue class inheriting soul::patch::VirtualFile, which simply reads the data from member choc::value::Value in read() call:



    …and I was hoping I could use it with my ExternalDataProvider to supply data for those external variables in Soul patch.



    …but turns out it doesn’t work this way, compileNewPlayer() function returns an error that it can’t read the file.

Thereby a few assumptions arise:

  • the intention for the ExternalDataProvider only to be used for audio files.
  • external variables for a soul patch are also intended to be used only for audio files if you’re using your own ExternalDataProvider. Or the use-case for constant values is not implemented.

I’ve been digging into the Externals and ExternalDataProvider for quite a bit now and here’s an update on my finding:

My initial assumptions

  • Data for the external patch variables can be provided from the host
  • You can provide sample data for external sample data (audio file) variables from the host

What it actually appears to be

  • External variables that are not audio files are resolved by the compiler/linker from the data you’ve added to the manifest file.

  • For a non-array, external primitives you can use annotation to initialize default value

  • External audio file variables are resolved by the compiler/linker from the files located in the same folder as the manifest file and they are loaded, read, mixed and resampled into choc::value::Value objects.

  • Your own ExternalDataProvider can be used to provide file location. Current implementation tries to read everything you provide as an audio file. So this is helpful to resolve file paths, in case they are located not in the same location as the manifest file. Or you can write a VirtualFile with soul::AudioFileFactory::createMemoryDataSource() to provide a virtual file that reads from memory. But the program is still going to load sample data from it (as far as I understand), so it’s not much use for memory management.


    (soul_patch_PlayerImpl.h)

  • A huge limitation is that the audio files can’t be longer than 60 seconds, otherwise AudioFileToValue used in PlayerImpl fails to load them into memory as choc::value::Value and currently there’s no mechanism for streaming from disk.


For my project it's absolutely crucial to manage loading and decoding sample data in and out of memory as needed. Therefore I still have some digging to do to figure out how to provide large chunks of sample data instead of file paths.

I hope I won’t have to copy and rewrite PlayerImpl and InstanceImple to hook into the resolveExternalVariable() function and be able to load externals directly as choc::value::Value instead of audio files. I’m not sure if that’s even possible, I hope I won’t have to go down that rabbit hole :sweat_smile:

My goal with this adventure is to be able to point to a chunk of memory owned by the host for patch to read sample data from.