OK, so here are more details based on what I was able to get working.
There are a few disclaimers to this however:
- I wasn’t really able to get the plugin GUI editor working in Unity (but that wasn’t a huge issue for me, so I didn’t spend too much time on it)
- In my design I just needed a single instance of the plugin, and my code reflects this. If you need to be able to create multiple instances of your plug-in in Unity, you will probably need to modify the approach slightly to properly instantiate and destroy plug-in instances in your Unity Native Audio SDK project (this will make more sense later after seeing some of the example code, but you can check out how JUCE does this for the generated Unity project files)
- I used nmake and the Android NDK (which Unity was using) to cross compile the audio plug-in
- Because I was wary of potential JUCE dependency issues cross compiling to Android, I explicitly was able to abstract the core of my plugin and DSP code to portable C++ that had no specific JUCE dependencies, but for the normal macOS/Win JUCE Unity plugin it was wrapped in a JUCE AudioProcessor. Unfortunately, I can’t really provide much insights with how well this will work if you have a lot of JUCE dependencies.
With that out of the way, I was able to find a post on the Unity forums in which people were asking about Native Audio SDK for Anrdoid (since was not officially supported), and found a link to this project which successfully got a Native Audio SDK plugin working on Anrdoid:
https://github.com/playdots/UnityPd.
This is not a JUCE project, but it gave clues on how to setup the project structure.
I created normal JUCE Unity audio plugin projects with Projucer for macOS and Windows for testing during development in Unity which produce a .bundle and .dll, respectively. And then for the Android version of the audio plugin, I had to manually create the project files for use with nmake to generate the .so file.
I’m going to assume you have familiarized yourself with how Unity Native Audio SDK plugins work (https://docs.unity3d.com/Manual/AudioMixerNativeAudioPlugin.html) and (https://github.com/Unity-Technologies/NativeAudioPlugins).
The short summary is that there is a separate C++ project template/SDK Unity supplies to use as a starting point to implementing your own (in this case, non-JUCE) native audio plugins. This is what the JUCE Unity audio plugin projects are generating. But for Android this is not getting generated, as you probably know. Unity also was not officially supporting native audio plugins for Android (not sure if they do now), but it is possible to make them work.
I used the UnityPd project I mentioned above as a starting point since I know that it was a working implementation of Native Audio plugins on Android. So, some of the files I will post as an example will still have some leftover UnityPd related values that I was too lazy to change.
Basically my project structure looked like this (I’m going to substitute my actual plugin name with MyPlugin, also I’m omitting some files like licenses, and just trying to list the critical ones):
- [ Root ]
- [ app ]
- [ jni ]
- Android.mk
- Application.mk
- Android.iml
- [ UnityNativeAudioPlugin ]
- AudioPluginInterface.h
- AudioPluginUtil.cpp
- AudioPluginUtil.h
- PluginList.h
- Plugin_MyPlugin.cpp
To edit the Makefile for building, edit /jni/Android.mk.
To edit the target archteciture, edit /jni/Application.mk.
To build, set your current/working directory to the /jni folder and make sure ndk-build is in your env PATH variable. You can use ndk-build clean, ndk-build DEBUG=true, and ndk-build, to clean, build in Debug mode, and build in Release mode respectively.
The Unity Native Audio SDK interface is implemented in Plugin_MyPlugin.cpp, and the UnityNativeAudioPlugin directory. The plugin is registered as a Unity audio plugin in PluginList.h.
Plugin_MyPlugin.cpp wraps your audio class as a Unity audio plug-in.
PluginList.h registers the plugin by name and C++ namespace within Unity’s audio plug-in system:
DECLARE_EFFECT("audioplugin_MyPlugin", Plugin_MyPlugin)
Make sure your name starts with “audioplugin_” as I believe Unity uses this as a convention when scanning for audio plugin libraries.
Here’s where you should place each operating systems dynamic library in your Unity project:
Windows: ~/Assets/Plugins/{{x86, x64}}/audioplugin_MyPlugin.dll
macOS: ~/Assets/Plugins/macOS/audioplugin_MyPlugin.bundle
Android ~/Assets/Plugins/Android/{{arm64-v8a, armeabi-v7a}}/audioplugin_MyPlugin.so
(the stuff in the curly braces is meant to indicate that you need to create a new folder per architecture you want to target and place the respective library binary in those folders for Unity to find it for each platform you want to target)
I ended up adding an C/C++ interop interface to my audio plugins so that I could use C# interop to interact with my plugin in C# and call underlying C++ methods which worked well, but I’m not going to get into that here.
Here us the contents of the files (as much as I can provide, again some code omitted since it is project-specific, and not open source):
/app/app.iml
<?xml version="1.0" encoding="UTF-8"?>
<module type="JAVA_MODULE" version="4">
<component name="NewModuleRootManager" inherit-compiler-output="true">
<exclude-output />
<orderEntry type="sourceFolder" forTests="false" />
</component>
</module>
/jni/Android.mk
LOCAL_PATH := $(call my-dir)
# Setup your source paths, these are mine, just as an example
CYSO_JUCE_ROOT = $(LOCAL_PATH)/../../../../../../CySoJuce/src/modules
#DST_PATH = $(LOCAL_PATH)/../../../../build
# PD-specific flags
PD_SRC_FILES := \
$(CYSO_JUCE_ROOT)/cymatic_somatics_filters/utility/WindowFunction.cpp \
$(CYSO_JUCE_ROOT)/cymatic_somatics_filters/filters/CySoButterworthFilter.cpp \
# Add your source files here, I put some of mine in here as an example ...
(CYSO_JUCE_ROOT)/cymatic_somatics_my_plugin/MyPlugin.cpp
PD_C_INCLUDES := $(CYSO_JUCE_ROOT)/cymatic_somatics_math/math \
# Add your source include directories here, again the ones here are just for example ...
$(CYSO_JUCE_ROOT)/cymatic_somatics_my_plugin/myplugin
PD_CFLAGS := -DPD -DHAVE_UNISTD_H -DHAVE_LIBDL -DUSEAPI_DUMMY -w
PD_JNI_CFLAGS := -Wno-int-to-pointer-cast -Wno-pointer-to-int-cast
PD_LDLIBS := -ldl
# build static library
include $(CLEAR_VARS)
LOCAL_MODULE := MyPlugin
LOCAL_C_INCLUDES := $(PD_C_INCLUDES)
LOCAL_CFLAGS := $(PD_CFLAGS)
LOCAL_LDLIBS := $(PD_LDLIBS)
LOCAL_SRC_FILES := $(PD_SRC_FILES:$(LOCAL_PATH)/%=%)
LOCAL_EXPORT_C_INCLUDES := $(PD_C_INCLUDES)
include $(BUILD_STATIC_LIBRARY)
# build plugin
include $(CLEAR_VARS)
LOCAL_MODULE := audioplugin_MyPlugin
LOCAL_C_INCLUDES := $(LOCAL_PATH)/../../UnityNativeAudioPlugin
LOCAL_C_FLAGS = -DPD
LOCAL_LDLIBS += -latomic
LOCAL_SRC_FILES := ../../Plugin_MyPlugin.cpp ../../UnityNativeAudioPlugin/AudioPluginUtil.cpp
LOCAL_STATIC_LIBRARIES := MyPlugin
include $(BUILD_SHARED_LIBRARY)
#all: $(DST_PATH)/$(TARGET_ARCH_ABI)
#$(DST_PATH)/$(TARGET_ARCH_ABI): $(LOCAL_BUILT_MODULE)
# mkdir -p $@ && cp $< $@
/jni/Application.mk (change your target architecture here)
APP_STL := c++_static
APP_ABI := armeabi-v7a arm64-v8a
NDK_TOOLCHAIN_VERSION := clang
/jni/Android.iml
<?xml version="1.0" encoding="UTF-8"?>
<module type="JAVA_MODULE" version="4">
<component name="NewModuleRootManager" inherit-compiler-output="true">
<exclude-output />
<orderEntry type="sourceFolder" forTests="false" />
</component>
</module>
/UnityNativeAudioPlugin/AudioPluginInterface.h:
https://github.com/Unity-Technologies/NativeAudioPlugins/blob/master/NativeCode/AudioPluginInterface.h
/UnityNativeAudioPlugin/AudioPluginUtil.cpp
https://github.com/Unity-Technologies/NativeAudioPlugins/blob/master/NativeCode/AudioPluginUtil.cpp
/UnityNativeAudioPlugin/AudioPluginUtil.h
https://github.com/Unity-Technologies/NativeAudioPlugins/blob/master/NativeCode/AudioPluginUtil.h
/UnityNativeAudioPlugin/PluginList.h (part of the Unity Native Audio SDK)
DECLARE_EFFECT("audioplugin_PrismDecoder", Plugin_MyPlugin)
/Plugin_MyPlugin.cpp
#include "AudioPluginUtil.h"
// These are my custom includes, just putting them in here as an example
#include "CySoMath.h"
#include "WindowFunction.h"
/*
Your audio plugin includes here, etc.
*/
using namespace CymaticSomatics;
namespace Plugin_MyPlugin // this namespace is important as I believe Unity's Native Audio SDK uses it
{
// Here's my single instance, you might need a different approach if you need multiple instances of your plugin.
MyPlugin myPlugin;
enum Param
{
P_NUM
};
/*
This is normally where you setup your plugins parameter data.
If you need to support more than once instance of your plugin, I believe you
can add a pointer to it to this structure, with life cycle explained in the comments below.
I would recommend looking at the JUCE Unity audio plugin generated files to see how they do it.
*/
struct EffectData
{
struct Data
{
float p[P_NUM];
};
union
{
Data data;
unsigned char pad[(sizeof(Data) + 15) & ~15]; // This entire structure must be a multiple of 16 bytes (and and instance 16 byte aligned) for PS3 SPU DMA requirements
};
};
/**
GUI CREATION
*/
int InternalRegisterEffectDefinition(UnityAudioEffectDefinition& definition)
{
int numparams = P_NUM;
definition.paramdefs = new UnityAudioParameterDefinition[numparams];
definition.channels = 2;
return numparams;
}
/**
THE SETUP EVENT
*/
UNITY_AUDIODSP_RESULT UNITY_AUDIODSP_CALLBACK CreateCallback(UnityAudioEffectState* state)
{
EffectData* effectdata = new EffectData;
memset(effectdata, 0, sizeof(EffectData));
state->effectdata = effectdata;
// Initialize your audio plugin here...
/*
If you need multiple instances of your plugin, instantiate them here I believe.
Add a pointer to them on your EffectData structure so you can delete them later.
*/
InitParametersFromDefinitions(InternalRegisterEffectDefinition, effectdata->data.p);
return UNITY_AUDIODSP_OK;
}
UNITY_AUDIODSP_RESULT UNITY_AUDIODSP_CALLBACK ReleaseCallback(UnityAudioEffectState* state)
{
EffectData::Data* data = &state->GetEffectData<EffectData>()->data;
// If you need multiple instances of your plugin, make sure to delete them here (you should at a pointer to them on the EffectData structure).
delete data;
return UNITY_AUDIODSP_OK;
}
UNITY_AUDIODSP_RESULT UNITY_AUDIODSP_CALLBACK SetFloatParameterCallback(UnityAudioEffectState* state, int index, float value)
{
EffectData::Data* data = &state->GetEffectData<EffectData>()->data;
if (index < 0 || index >= P_NUM)
return UNITY_AUDIODSP_ERR_UNSUPPORTED;
data->p[index] = value;
return UNITY_AUDIODSP_OK;
}
UNITY_AUDIODSP_RESULT UNITY_AUDIODSP_CALLBACK GetFloatParameterCallback(UnityAudioEffectState* state, int index, float* value, char* valuestr)
{
EffectData::Data* data = &state->GetEffectData<EffectData>()->data;
if (index < 0 || index >= P_NUM)
return UNITY_AUDIODSP_ERR_UNSUPPORTED;
if (value != NULL)
*value = data->p[index];
if (valuestr != NULL)
valuestr[0] = 0;
return UNITY_AUDIODSP_OK;
}
int UNITY_AUDIODSP_CALLBACK GetFloatBufferCallback(UnityAudioEffectState* state, const char* name, float* buffer, int numsamples)
{
return UNITY_AUDIODSP_OK;
}
UNITY_AUDIODSP_RESULT UNITY_AUDIODSP_CALLBACK ProcessCallback(UnityAudioEffectState* state, float* inbuffer, float* outbuffer, unsigned int length, int inchannels, int outchannels)
{
/*
Run your audio processor's processBlock() method here.
Keep in mind you must deinterlace the audio buffers from Unity first,
since Unity uses interlaced audio buffers and JUCE does not. Don't
forget to go back to interlaced audio before copying to the outbuffer for Unity.
*/
// Copy the input buffer to the output buffer
if (inchannels >= 2 && outchannels >= 2)
memcpy(outbuffer, inbuffer, sizeof(float) * length * 2);
return UNITY_AUDIODSP_OK;
}
}
I know this is not a complete working example, but hopefully it gets you a bit closer. You probably can’t avoid all JUCE dependencies like I did, but hopefully you can get them to cross compile OK. I modified the source to be appropriate for sharing an example, so not sure if I messed anything up when doing that, but I tried to leave some helpful comments of useful things to know.
Cheers, and good luck.
