Streaming audio from one VST to another over sockets

Hello,

I’m trying to build a plugin that can send audio from one DAW to another without any modification and I had a few questions and could use some advice :slight_smile: . The idea is the remote computer can hit play and the host should be able to hear the audio.

I’ve created the plugin and I’m able to get both plugins to connect using StreamingSockers, which is cool, but now I need to actually send the audio. What would be the most viable approach? I was thinking, in the processBlock to use the connected socket to transfer the channelData from the remote to the host and then just overwrite the hosts channelData. SO if both are pressing play at the same time in theory it would play the remote connections audio into the host computer.

Does this make sense? Is there a different way to do this? If not, im a bit lost as to how to convert the channelData into something that can be passed to the socket to write.

Any advice or info would be really appreciated thank you!!

image

You don’t want to do anything blocking, or possibly time consuming, in processBlock. This is a “realtime” context, and you can easily cause the audio to glitch because you took too long. Your best bet is to copy the audio data in processBlock and use a different thread to do the network communications… This will still have issues that you need to work out, in how do you copy the data and alert the other thread, in a way that is non-blocking, and doesn’t do things like allocate memory.

I wrote two plugins like that back in the day. apulSoft Wormhole and plasq Wormhole2. The later is still available somewhere as open-source. It did send audio over udp sockets and just sent the raw float samples with some timestamps. It’s kind of easy to get this to work in a basic way, but it’s a terrible problem to solve if you want to do it right. The problems are latency and timestamps.
The biggest issue is that there is no way to get a sample accurate reference clock for two machines. Another issue is that even if both machines run at the same sample rate, their rates will drift over time which means some form of resampling would be needed to stay in time. Additionally, network communication using sockets is not designed to produce reliable timing. There might be packet loss and/or delay at any time which is terrible for this use-case. Buffering can make these issues less of a problem, but the big issue of never being able to totally sample-lock networked machines remains. The problem gets simpler if it is a round-trip situation as then the delay can be accurately measured and the sample clock of the other machine might not even matter - it just becomes a sample cruncher.

What you are reporting about drifting clocks seems quite obvious. However, as there are networked realtime audio over IP protocols like AES67, AVB, Dante, there seem to be solutions for that. Not sure if there is some ready available AES67 implementation one could use for this?

There is a free AES67 driver by merging technologies. However these protocols include syncing the sample clock over ethernet which is required to get rid of the issues I described, but is something that is totally outside the scope of what an audio plugin can do - it needs system access on driver level.

Thanks everyone for the insightful comments! I figured real-time would have latency delays but sounds tougher than I imagined. Recording it as a sample, sending it back, and then letting the host play it as a sample seems like a way to accomplish this goal without too many technical headaches.