Audio Streaming via IPC

I wonder what would be the best way to merge the audio output of multiple processes (programs) into one, without using Jack, SoundFlower, or similar third party tools.

The idea is to have a number of host processes that feed their outputs to a master process, which drives local audio hardware.

This could be used as a 32/64 bit bridge (2 hosts on same computer), or for load balancing (multiple slave hosts on LAN, one master on local computer). One of the hosts would take the master role. Other hosts would select that master for output, preferably on their audio settings dialog. 

Of course, since all streams represent the same timeline, they need to be synchronized in some way (timestamped), which probably adds to the latency.

Would that work with Juce IPC over IP? Pipes? Shared memory? What would be the most simple "architecture" to implement this? Has anyone done it before? How would one best go about it without going crazy? 

Any pointers and hints are much appreciated. 
Thanks in advance!