So I am about to embark on a project with the following goal:
Enable the user to script audio synthesis/filtering algorithms in Python/Numpy and hear the results in real-time via a VST hosted in a DAW.
Think of it as Chuck or MAX/MSP wrapped in a VST (ala JUCE of course) but using Python/Numpy instead. Somehow I doubt I was the first one to think of doing something like this, so a question to the giants who came before:
Is it better to embed the Python interpreter in the plugin? or should I extend Python with a module that communicates (via IPC? ReWire?) with the plugin which would just act as a sort of sound server. Based on my research so far, it seems much easier to extend Python in that you don’t have to do any extra work to maintain the Python readline/event loop and so the user can use his own Python environment. However, I don’t have experience with low-latency audio over an IPC channel.
Python is rather hard to embed (compared to javascript or lua), and the documentation explicitely encourages extension.
There have been attempts to host and write VST with python and numpy, but keep in mind that for serious audio work, this won’t meet real-time specifications because of the following 3 reasons: Python’ Global Interpreter Lock, Memory allocation and Garbage collection that have unbounded execution time.
With respect to embed vs extend you have succinctly summarized the result of my research into the subject. However, I am still hoping for someone with actual war stories about embedding Python would step forward. For example, isn’t Python embedded into Ableton Live and Blender to great effect?
I certainly agree that its not appropriate to have Python generate audio in real-time especially if it is generating/calculating each sample. Rather I wish to provide functional blocks (written in C/C++) that can be controlled and connected via Python. So Python will operate only in the control path not the audio path (which will be pure C/C++). I imagine this is how a lot of interactive audio programming environments (like MAX/MSP) are designed anyways.
Still having said that, I think you are right about identifying garbage collection as an annoyance (hopefully not a show stopper) that I will have to deal with even in the Python-only-in-the-control-path scenario.
