# Unity3D Game Engine with JUCE Audio Backend **URL:** \ **Category:** General JUCE discussion\ **Created:** [February 17, 2017, 9:35pm UTC](https://forum.juce.com/t/unity3d-game-engine-with-juce-audio-backend/21019 "2017-02-17T21:35:58Z")\ **Posts on this page:** 15\ **Page:** 1
**Author:** ![cpenny42](https://sea2.discourse-cdn.com/flex026/user_avatar/forum.juce.com/cpenny42/32/1281_2.png) [@cpenny42](https://forum.juce.com/u/cpenny42)\ **Post date:** [February 17, 2017, 9:35pm UTC](https://forum.juce.com/t/unity3d-game-engine-with-juce-audio-backend/21019/1 "2017-02-17T21:35:58Z")
Does anyone have any experience with this? I’m about to jump headfirst into this rabbit hole. My end goal is to target AR & VR (Augmented Reality & Virtual Reality) apps while being able to leverage the power of the JUCE audio APIs. My initial research suggests that it’s not as simple as I would have thought to use C++ from within Unity3D (Unity is made with C++, but most of the work is done in C# via the Mono engine). I’ve developed a simple tool (basically a simple transpiler) that converts my audio backend (using JUCE obviously) into a simple C API accessible through a Header & a static library with no other dependencies — this has worked wonders for using JUCE as an audio backend for mobile apps and other projects where JUCE is used solely for the Audio bits and everything else is done in the other programming language. From the basic C API, it’s pretty simple to auto-generate other language bindings — so far I’ve done Objective-C, Swift, Java, & Pure Data, and I’m working on the Julia, Python,& Javascript bindings (Javascript via Emscripten — a whole can of worms in itself). My initial impression would be that it would be best to generate a C# API built off the C API and make a Unity3D plugin from that — it’s possible to link static & dynamic libraries in Unity, and I think having the C# bindings would make things simpler. Does anyone have any experience with this sort of thing? I understand it’ll be a bit of work no matter what, but I really feel like it would be worth it to develop a solid workflow that allows us to leverage both of these powerful tools together. If we are successful, this could be a godsend for developers everywhere. FWIW, I plan on open-sourcing my JUCE -\> C transpiler soon (as well as C -\> Other Languages), but it’s nothing fancy so far, just a simple Lex & Parser engine. I want to switch from my own home-brewed system to using the LLVM Lexing & Parsing tools, but in the meantime I have something that works pretty well, and I have a feeling that it could actually be pretty simple to hook up the raw C API into Unity3D. I just barely started using Unity3D, though, so I could use all the advice I can get! Thanks in advance! ---
**Author:** ![daniel](https://sea2.discourse-cdn.com/flex026/user_avatar/forum.juce.com/daniel/32/790_2.png) [@daniel](https://forum.juce.com/u/daniel)\ **Post date:** [February 17, 2017, 11:41pm UTC](https://forum.juce.com/t/unity3d-game-engine-with-juce-audio-backend/21019/2 "2017-02-17T23:41:30Z")
I haven’t found a link to the sources of Unity3d (just quick browsing), but what I saw in the documentation it seems like it uses (also) OpenAL? If so I would create a subclass of AudioIODevice which calls the callback of the [AudioIODeviceCallback](https://www.juce.com/doc/classAudioIODeviceCallback) to fill an OpenAL buffer. I suggested that already here (but had no time to make a proof of concept): > [@Has anyone integrated JUCE within a game built on Unreal Engine?](https://forum.juce.com/t/has-anyone-integrated-juce-within-a-game-built-on-unreal-engine/20105/3): > > I am looking into this now. My thoughts are, there is support for an audio stream. So if somebody would add an AudioOutputStream as subclass of [AudioIODeviceCallback](https://www.juce.com/doc/classAudioIODeviceCallback), that would be awesome. The other option would be to hook directly into OpenAL, which is the audio backend of Unreal. How buffers are queued is described on page 14: [http://openal.org/documentation/OpenAL\_Programmers\_Guide.pdf](http://openal.org/documentation/OpenAL_Programmers_Guide.pdf) I would be interested, how it went… Or did you actuall want to go the other way round and feed the game sound into JUCE buffers? ---
**Author:** ![cpenny42](https://sea2.discourse-cdn.com/flex026/user_avatar/forum.juce.com/cpenny42/32/1281_2.png) [@cpenny42](https://forum.juce.com/u/cpenny42)\ **Post date:** [February 17, 2017, 11:55pm UTC](https://forum.juce.com/t/unity3d-game-engine-with-juce-audio-backend/21019/3 "2017-02-17T23:55:24Z")
That actually sounds perfect! Really I just wanted to use JUCE as the audio engine, most of the work would be in Unity. I think it would be awesome to develop virtual reality apps with JUCE doing the audio. I’ll update with how it works out. ---
**Author:** ![rory](https://sea2.discourse-cdn.com/flex026/user_avatar/forum.juce.com/rory/32/534_2.png) [@rory](https://forum.juce.com/u/rory)\ **Post date:** [February 18, 2017, 12:45pm UTC](https://forum.juce.com/t/unity3d-game-engine-with-juce-audio-backend/21019/4 "2017-02-18T12:45:21Z")
> [@cpenny42](#): > > Really I just wanted to use JUCE as the audio engine Why use JUCE as the audio engine when Unity has its own audio engine? My advice would be to use Unity to handle audio IO and then send it to your JUCE-based binary through Unity’s OnAudioFilterRead() method. ``` void OnAudioFilterRead(float[] data, int channels) { yourBinaryObject.processBlock(data, channels); } ``` ---
**Author:** ![mayae](https://avatars.discourse-cdn.com/v4/letter/m/9fc29f/32.png) [@mayae](https://forum.juce.com/u/mayae)\ **Post date:** [February 18, 2017, 1:42pm UTC](https://forum.juce.com/t/unity3d-game-engine-with-juce-audio-backend/21019/5 "2017-02-18T13:42:58Z")
You want to do everything in your power to avoid OnAudioFilterRead - since it runs in the C# domain, the thread is subject to garbage collection. There are multiple other ways to directly do what you want - Native plugins, audio mixer plugins, spatializer plugins… Note these are all “native” and you can thus use C++ (and JUCE) directly from Unity3D. ---
**Author:** ![rory](https://sea2.discourse-cdn.com/flex026/user_avatar/forum.juce.com/rory/32/534_2.png) [@rory](https://forum.juce.com/u/rory)\ **Post date:** [February 18, 2017, 5:34pm UTC](https://forum.juce.com/t/unity3d-game-engine-with-juce-audio-backend/21019/6 "2017-02-18T17:34:05Z")
Hi Mayae, by writing a native plugin won’t you have to implement your own audio IO routines? I did this in the past but found it quite cumbersome as my library and Unity would compete for the same audio IO. I could disable Unity’s audio IO, but it’s pretty good and finding and selecting the default devices. I guess JUCE is just as capable of doing this. Anyhow, nowadays I create a class derived from AudioSource and send samples too and fro an external processing library. I’ve never had any issues with drop-outs or latency, and I’ve pushed it pretty hard. It also means I can use my custom audio objects alongside Unity’s. Afaik there is no way to do this without calling OnAudioFilterRead()? Or maybe there is, I’m certainly no expert. ---
**Author:** ![mayae](https://avatars.discourse-cdn.com/v4/letter/m/9fc29f/32.png) [@mayae](https://forum.juce.com/u/mayae)\ **Post date:** [February 18, 2017, 7:01pm UTC](https://forum.juce.com/t/unity3d-game-engine-with-juce-audio-backend/21019/7 "2017-02-18T19:01:19Z")
I mean the native plugins inside Unity. Unity can load your DLLs, SOs or bundles. You can then either interface yourself or you can use the native audio plugins SDK (a C interface for integrating your audio routines). Note this is all happening inside Unity, it’s not external. It’s not so much what you do inside OnAudioFilterRead, it’s what you do on the outside - loading lots of assets, creating objects each frame etc. will trigger the GC every so often, and it will impact the performance and headroom of the procedural audio generation. Edit: Have a look: [https://docs.unity3d.com/500/Documentation/Manual/AudioMixerNativeAudioPlugin.html](https://docs.unity3d.com/500/Documentation/Manual/AudioMixerNativeAudioPlugin.html) ---
**Author:** ![rory](https://sea2.discourse-cdn.com/flex026/user_avatar/forum.juce.com/rory/32/534_2.png) [@rory](https://forum.juce.com/u/rory)\ **Post date:** [February 18, 2017, 7:04pm UTC](https://forum.juce.com/t/unity3d-game-engine-with-juce-audio-backend/21019/8 "2017-02-18T19:04:33Z")
Ah yes, now I follow. I also tried this route before, and their SDK is pretty easy to use. But the last I tried it was only for audio effects. In my case I was writing generate stuff and it just wasn’t intuitive to use Unity’s native SDK. But for what @cpenny42 it could very well be a good solution. ---
**Author:** ![mnsp](https://sea2.discourse-cdn.com/flex026/user_avatar/forum.juce.com/mnsp/32/2665_2.png) [@mnsp](https://forum.juce.com/u/mnsp)\ **Post date:** [August 21, 2017, 5:24pm UTC](https://forum.juce.com/t/unity3d-game-engine-with-juce-audio-backend/21019/9 "2017-08-21T17:24:13Z")
The ideal solution would be if we could generate Unity native audio plugins using JUCE. I understand that the GUI would have to be written separately in C#, but the rest of the plugin interface seems quite similar to VST and others, so in principle that should be possible. ---
**Author:** ![Nico\_Raftis](https://sea2.discourse-cdn.com/flex026/user_avatar/forum.juce.com/nico_raftis/32/3212_2.png) [@Nico\_Raftis](https://forum.juce.com/u/Nico_Raftis)\ **Post date:** [May 11, 2018, 2:58am UTC](https://forum.juce.com/t/unity3d-game-engine-with-juce-audio-backend/21019/10 "2018-05-11T02:58:02Z")
Yes, Unity has a great native audio SDK that would be ideal as a JUCE export option. The C# creates Unity Editor controls that work with other aspects of the game engine scripting API, a few really good open source references are : [https://github.com/mtytel/helm](https://github.com/mtytel/helm) - a full JUCE based synthesizer with a unity asset that can load a binary of the audio engine and control the synth (the unity asset is 80 dollars though and I think might be closed source) a reference for how this works in practice is in his video tutorials [https://www.youtube.com/watch?v=1oNWX0igFMo&index=3&list=PLyQHewKAoFRKldoPGM6foug1g7RZTkPeO](https://www.youtube.com/watch?v=1oNWX0igFMo&index=3&list=PLyQHewKAoFRKldoPGM6foug1g7RZTkPeO) [https://github.com/grame-cncm/faust/tree/master-dev/architecture/unity](https://github.com/grame-cncm/faust/tree/master-dev/architecture/unity) - faust2unity generates a binary and C# script to control it, but I filed a few caveats about the nature of their script interface that I am helping sort. and my favorite is Heavy compiler for Pure Data: [https://enzienaudio.com/docs/index.html#06.unity](https://enzienaudio.com/docs/index.html#06.unity) this compiles pure data patches into VST, Unity Native Audio binary / C# script for interface and many other formats including C++ source code. I think what you need to do is mostly look at the examples for the unity native audio SDK and think of that like the way JUCE treats the VST SDK, create a C# script interface generator that facilitates a connection to the compiled binary from JUCE [https://docs.unity3d.com/Manual/AudioMixerNativeAudioPlugin.html](https://docs.unity3d.com/Manual/AudioMixerNativeAudioPlugin.html) ---
**Author:** ![ed95](https://sea2.discourse-cdn.com/flex026/user_avatar/forum.juce.com/ed95/32/7635_2.png) [@ed95](https://forum.juce.com/u/ed95)\ **Post date:** [May 11, 2018, 5:21am UTC](https://forum.juce.com/t/unity3d-game-engine-with-juce-audio-backend/21019/11 "2018-05-11T05:21:06Z")
> [@Experimental support for Unity native audio plugins on the develop branch](https://forum.juce.com/t/experimental-support-for-unity-native-audio-plugins-on-the-develop-branch/27621): > > With commit [527625b](https://github.com/WeAreROLI/JUCE/commit/527625b699dda6cc9597b96b264da270cc098cc8), JUCE now supports building Unity native audio plugins for Windows and Mac standalone builds. This format can be enabled like any other plugin format in your project settings in the Projucer: The plugin will be compiled to a .bundle on macOS and a .dll on Windows which can then be imported and used in the Unity editor and standalone builds of your game. The Projucer will also create a C# script (generated in JuceLibraryCode) to handle your plugin’s GUI and this will b… ---
**Author:** ![cpenny](https://sea2.discourse-cdn.com/flex026/user_avatar/forum.juce.com/cpenny/32/4662_2.png) [@cpenny](https://forum.juce.com/u/cpenny)\ **Post date:** [May 17, 2018, 2:47pm UTC](https://forum.juce.com/t/unity3d-game-engine-with-juce-audio-backend/21019/12 "2018-05-17T14:47:59Z")
Nice! This is what I love about JUCE. ---
**Author:** ![nammick](https://sea2.discourse-cdn.com/flex026/user_avatar/forum.juce.com/nammick/32/2169_2.png) [@nammick](https://forum.juce.com/u/nammick)\ **Post date:** [June 6, 2018, 9:15am UTC](https://forum.juce.com/t/unity3d-game-engine-with-juce-audio-backend/21019/13 "2018-06-06T09:15:54Z")
Depending on what you want to do. You can attach an audio filter to a sprite and access the raw data > void OnAudioFilterRead(float data, int channels) > { /\* process the data like in Juce \*/ } We built a whole project round this in the past and works very well. ---
**Author:** ![DaveH](https://sea2.discourse-cdn.com/flex026/user_avatar/forum.juce.com/daveh/32/4924_2.png) [@DaveH](https://forum.juce.com/u/DaveH)\ **Post date:** [June 11, 2018, 4:37pm UTC](https://forum.juce.com/t/unity3d-game-engine-with-juce-audio-backend/21019/14 "2018-06-11T16:37:19Z")
You can call any native DLL using the OnAudioFilerRead function. The slight headache comes when sending parameters to the native code. How did you accomplish that smoothly? I’ve found it a pain in the past and just resorted to floating point blocks of data. ---
**Author:** ![nammick](https://sea2.discourse-cdn.com/flex026/user_avatar/forum.juce.com/nammick/32/2169_2.png) [@nammick](https://forum.juce.com/u/nammick)\ **Post date:** [June 11, 2018, 5:25pm UTC](https://forum.juce.com/t/unity3d-game-engine-with-juce-audio-backend/21019/15 "2018-06-11T17:25:14Z")
Well in this use case we were just using that function in Unity to generate sine tones and lfo warbles for gamifing audiometry for kids [https://clik.audio](https://clik.audio) But I did a flash DJ app a few years ago with sockets in the same using flex app as a front end and a little juce app as a backend. It was purely for my own education but it worked really well + sockets are easy to work with in unity. Just need to design a simple protocol to do something (key pair / json / xml or just raw bytes enumerated)