Patch for adding MIDI Output support to AudioUnit effects


#1

Attached is a patched version of juce_AU_Wrapper.mm that allows for AudioUnit effects to send MIDI out. This patch has been verified to work in JUCE Plugin Host. Added .txt extension to the filename in order to be able to attach it to this post.

I am by no means an expert on the details of the AudioUnit API or JUCE audio processing, so this patch might need additional work.

This patch is mostly based upon this AudioUnit example from Apple named SinSynthWithMidi: https://developer.apple.com/library/mac/samplecode/SinSynth/Listings/SinSynthWithMidi_cpp.html

Not all details will be covered in this post, only the significant ones.

Using JUCE Plugin Host for debugging I saw that AudioUnitPluginInstance checks if an AudioUnit produces midi output in AudioUnitPluginInstance::canProduceMidiOutput and sets up the MIDI Output callback in AudioUnitPluginInstance::setPluginCallbacks(). These functions use the AudioUnit properties kAudioUnitProperty_MIDIOutputCallback and kAudioUnitProperty_MIDIOutputCallbackInfo to perform these actions.

In juce_AU_wrapper.mm I added handling of these properties in the JuceAU class as done in the SinSynthWithMidi sample.

    ComponentResult GetPropertyInfo (AudioUnitPropertyID inID,
                                     AudioUnitScope inScope,
                                     AudioUnitElement inElement,
                                     UInt32& outDataSize,
                                     Boolean& outWritable) override
    {
        if (inScope == kAudioUnitScope_Global)
        {
#if JucePlugin_ProducesMidiOutput
            if (inID == kAudioUnitProperty_MIDIOutputCallbackInfo)
            {
                outDataSize = sizeof(CFArrayRef);
                outWritable = false;
                return noErr;
            }
            else if (inID == kAudioUnitProperty_MIDIOutputCallback)
            {
                outDataSize = sizeof(AUMIDIOutputCallbackStruct);
                outWritable = true;
                return noErr;
            }
            else if(inID == kAudioUnitProperty_MIDIOutputCallbackInfo)
            {
                outDataSize = sizeof(AUMIDIOutputCallbackStruct);
                outWritable = false;
                return noErr;
            }
            else
#endif
    ...

    



    ComponentResult GetProperty (AudioUnitPropertyID inID,
                                 AudioUnitScope inScope,
                                 AudioUnitElement inElement,
                                 void* outData) override
    {
        if (inScope == kAudioUnitScope_Global)
        {
#if JucePlugin_ProducesMidiOutput
            if (inID == kAudioUnitProperty_MIDIOutputCallbackInfo)
            {
                CFStringRef strs[1];
                strs[0] = CFSTR("MIDI Callback");
                
                CFArrayRef callbackArray = CFArrayCreate(NULL, (const void **)strs, 1, &kCFTypeArrayCallBacks);
                *(CFArrayRef *)outData = callbackArray;
                return noErr;
            }
            else
#endif

   ...

    

ComponentResult SetProperty (AudioUnitPropertyID inID,
                                 AudioUnitScope inScope,
                                 AudioUnitElement inElement,
                                 const void* inData,
                                 UInt32 inDataSize) override
    {
#if JucePlugin_ProducesMidiOutput
        if (inScope == kAudioUnitScope_Global)
        {
            if (inID == kAudioUnitProperty_MIDIOutputCallback)
            {
                
                if (inDataSize < sizeof(AUMIDIOutputCallbackStruct)) return kAudioUnitErr_InvalidPropertyValue;
                
                AUMIDIOutputCallbackStruct *callbackStruct = (AUMIDIOutputCallbackStruct *)inData;
                
                if(callbackStruct)
                {
                    midiCallback = *callbackStruct;
                }
                
                return noErr;
                
            }
            
        }
        else
#endif
    ...

As can be seen in the last snippet of code I have added a midiCallback member to the JuceAU class by the type of AUMIDIOutputCallbackStruct.

Once these functions have been implemented an AU effect that produces MIDI now shows up with a MIDI Output pin in JUCE Plugin Host.

The last step is to actually send MIDI to the callback set in SetProperty above, this is done in JuceAU::ProcessBufferLists


    OSStatus ProcessBufferLists (AudioUnitRenderActionFlags& ioActionFlags,
                                 const AudioBufferList& inBuffer,
                                 AudioBufferList& outBuffer,
                                 UInt32 numSamples) override
    {
    
    ...

            if (! midiEvents.isEmpty())
            {
               #if JucePlugin_ProducesMidiOutput
                const juce::uint8* midiEventData;
                int midiEventSize, midiEventPosition;
                MidiBuffer::Iterator i (midiEvents);

                while (i.getNextEvent (midiEventData, midiEventSize, midiEventPosition))
                {
                    jassert (isPositiveAndBelow (midiEventPosition, (int) numSamples));


                    //xxx
                }
                
            if (! midiEvents.isEmpty())
            {
               #if JucePlugin_ProducesMidiOutput
                const juce::uint8* midiEventData;
                int midiEventSize, midiEventPosition;
                MidiBuffer::Iterator i (midiEvents);

                while (i.getNextEvent (midiEventData, midiEventSize, midiEventPosition))
                {
                    jassert (isPositiveAndBelow (midiEventPosition, (int) numSamples));


                    //xxx
                }
                
                if(midiCallback.midiOutputCallback)
                {
#if JUCE_IOS
                    const MIDITimeStamp timeStamp = mach_absolute_time();
#else
                    const MIDITimeStamp timeStamp = AudioGetCurrentHostTime();
#endif
                    MidiBuffer::Iterator i (midiEvents);
                    
                    MidiMessage midiMessage;
                    int midiSamplePos;
                    
                    const UInt32 numPackets = (UInt32)midiEvents.getNumEvents();
                    size_t dataSize = 0;
                    
                    // Get data size of MidiBuffer
                    while(i.getNextEvent(midiMessage, midiSamplePos))
                    {
                        dataSize += (size_t)midiMessage.getRawDataSize();
                    }
                    
                    MIDIPacket packet;
                    const size_t packetDataCapacity = sizeof (packet.data);
                    const size_t packetMembersSize = sizeof(MIDIPacket) - packetDataCapacity;
                    const size_t packetListMembersSize = (sizeof (MIDIPacketList) - packetDataCapacity);
                    
                    // Allocate MIDIPacketList to fit entire MidiBuffer
                    HeapBlock <MIDIPacketList> packetList;
                    packetList.malloc (packetListMembersSize + packetMembersSize*numPackets + dataSize, 1);
                    packetList->numPackets = numPackets;
                    
                    MIDIPacket* p = packetList->packet;
                
                    // Copy MidiMesages from MidiBuffer to MIDIPacketList
                    MidiBuffer::Iterator i2 (midiEvents);
                                                            
                    while(i2.getNextEvent(midiMessage, midiSamplePos))
                    {
                        const size_t size = (size_t)midiMessage.getRawDataSize();
                        p->timeStamp = timeStamp; // FIXME: Not sure if correct
                        p->length = size;
                        memcpy(p->data, midiMessage.getRawData(), size);
                        p = MIDIPacketNext (p);
                    }
                    
                    // Send MIDIPacketList
                    midiCallback.midiOutputCallback(midiCallback.userData, &audioTimeStamp, 0, packetList);
                }                
               #else
                // if your plugin creates midi messages, you'll need to set
                // the JucePlugin_ProducesMidiOutput macro to 1 in your
                // JucePluginCharacteristics.h file
                //jassert (midiEvents.getNumEvents() <= numMidiEventsComingIn);
               #endif

                midiEvents.clear();
            }

     ...

This last snippet converts a JUCE MidiBuffer to a MIDIPacketList and sends this packet to the MIDI Output callback set in the previous snippet of code.

The conversion from MidiBuffer to MIDIPacketList is based upon the code in MidiOutput::sendMessageNow in juce_mac_CoreMidi.cpp.

I am not sure if the timeStamp is correctly set and also if the second to last argument to midiOutputCallback named midiOutNum should be something else that a hardcoded 0.


#2

Awesome stuff!

I'll take a look through this as soon as I get a moment - in the meantime, any feedback about it from other people would be welcome!


#3

Just looking at this, I'm confused by this code:


            if (inID == kAudioUnitProperty_MIDIOutputCallbackInfo)
            {
                outDataSize = sizeof(CFArrayRef);
                outWritable = false;
                return noErr;
            }
            else if (inID == kAudioUnitProperty_MIDIOutputCallback)
            {
                outDataSize = sizeof(AUMIDIOutputCallbackStruct);
                outWritable = true;
                return noErr;
            }
            else if(inID == kAudioUnitProperty_MIDIOutputCallbackInfo)
            {

..the first and third if statements are identical, so the last block will never be called - I guess this is a typo, but it's not clear what you originally meant?


#4

Yeah, you're right, some code I forgot to remove.

The code should just be the first two if's.


        if (inID == kAudioUnitProperty_MIDIOutputCallbackInfo)
        {
            outDataSize = sizeof(CFArrayRef);
            outWritable = false;
            return noErr;
        }
        else if (inID == kAudioUnitProperty_MIDIOutputCallback)
        {
            outDataSize = sizeof(AUMIDIOutputCallbackStruct);
            outWritable = true;
            return noErr;
        }


Like in the SinSynthWithMidi sample.
 


#5

Ok, thanks.

Using the clock-on-the-wall time for the midi event timestamp can't be right - the events need to be given times that indicate their relative position in terms of the sample position in the current block of data. Looking at the Apple code that handles the incoming events and passes them to HandleMidiEvent(), it seems that the frame times are taken directly from the MIDIPacket::timeStamp member, so presumably that's how also the value that should be sent out...? (Although I also suspect that the value may just get mostly ignored)

Anyway, thanks for getting me rolling on this! I've checked in a version that seems to work now - feedback welcomed!


#6

Using the clock-on-the-wall time for the midi event timestamp can't be right - the events need to be given times that indicate their relative position in terms of the sample position in the current block of data. Looking at the Apple code that handles the incoming events and passes them to HandleMidiEvent(), it seems that the frame times are taken directly from the MIDIPacket::timeStamp member, so presumably that's how also the value that should be sent out...? (Although I also suspect that the value may just get mostly ignored)

I think this sounds more correct. I have little knowledge of the details of MIDI and even less of how it should be handled through these API's, which is why I wrote a comment in my patched version indicating that I was not sure how the timeStamp should be set.

Hopefully someone else will be able to help confirm the details, as I am not of much help here.