Sending Midi notes from audio plugin


#1

Quick info about myself:
I have some programming knowledge in java and C however I have only programmed simple robots, I am still a noob and the only knowledge I got is about basic loops. I heard about JUCE last night.

What I am trying to do:
Create a VST plugin that can run on mac and windows, as long as it works with Ableton live 10 that is more than enough. The plugin shall receive the audio stream and when the audio clip it shall send a midi note that can be received by a midi track. The plugin must have a latency less than 5ms.

What I have done so far:
I have installed JUCE, I have followed some kind of youtube tutorial and have created a gain plugin that works. Right now it is doing this

“channelData[sample] = buffer.getSample(channel, sample) * amountOfGain;”

to edit the sound level. I think I can put that in an if statement so

if (channelData[sample] = buffer.getSample(channel, sample) > “clipping”){

//Trigger midi note here

}

(I do not know the value for clipping, 256?)

I have searched the internet all over and I have looked a lot at the arpeggiator example plugin but I got stuck, I tried to use some parts of the code in my program but get “Use of undeclared identifier” for both midi and notes. I do not know where and how to declare those variables. I am stuck but I think this shouldn’t be too big of a task? I heard something about that it is not possible to send midi from an audio plugin without doing custom drivers and stuff? I got no idea how this works, all help is greatly appreciated!


#2

All audio processing is done in floats (in JUCE always, in most other DSP code as well). So clipping is above 1.0 and below -1.0.

That really depends, so we would need to know, which identifier you are talking about.
Often you find what you need in the arguments, in this case in processBlock() you found the audio data already. You have the midi buffer as the second argument, where you feed the midi note into.

However, this will make your MIDI accessible for plugins following on the track. It sounds to me like you want to send it to an arbitrary physical midi device outside your host? In this case you have to experiment a bit, but have a look at MidiOutput.
I would advise against it in a product, since it is a weird workflow where a normal user would struggle, but for in-house experiments it is worth a shot.


#3

Okay thank you, for now I am only going to send the midi to different devices within the host. I will take a look at that link


#4

If you’re only sending MIDI within the host then do it within processBlock. Use MidiOutput when you need more control - it allows you to send messages directly to devices, effectively bypassing the host.


#5

Okay I see, how do I send a simple midi message? I have tried using this row of code:

“auto message = MidiMessage::noteOn (1, 5, (uint8) 100);”

to send a midi message but I did not get it to work. I got my ableton midi channel configured to listen to midi from all inputs


#6

Did you add it to the midibuffer you are getting as argument to processBlock? Use MidiBuffer::addEvent() and set reasonable samplenumbers (the position inside the buffer)


#7

Sorry to high-jack this: How many events are pre-allocated? Don’t we run the risk of re-allocations if we add even a single event to the existing buffer? Aren’t memory-allocations a big No-No for realtime-audio?

Since the events are inserted, does that mean my plugin will receive the events that it just inserted itself? Is there a way to “mark” them, so they can be ignored?


#8

Yes, it is a No-no, but the MidiBuffer doesn’t do that. It uses the Array class, which reuses memory for trivially copyable objects, which is the case for MidiMessage, but it wouldn’t even have to, since the Array is actually an Array of uint8.

There is the ensureSize() to make room, I don’t know what the default size is though.

You are inserting the messages yourself, so what was in at the beginning of processBlock is, what came in and at the end, i.e. after your code, what goes out. If you want to replace the midi data, you will have to clear it yourself.

Hope that helps


#9

So reading the events, removes them from the buffer?

Also, the buffer is 2048 bytes (not events), which is good enough for 682 three-byte events.

So addEvent claims to insert the event at the correct position, so if I insert a new event at a position that is after (or between) incoming events, I would actually consume them myself and they would not be output to the host?

So the only viable/workable pattern would be to completely read it out first and then put my own events in before returning?

I’m not trying to be antagonistic here, just trying to understand how it works.


#10

No, default is you are adding to it. That’s why I said, if you want to replace them, you have to clear the buffer, i.e. remove the original events.

A typical audio buffer is max. 23 ms (1024 samples @ 44.1 kHz assumed worst case) how many events do you want to add in that time? :wink:

I am speaking theoretically here, didn’t measure anything of it myself. If possible, yes it sounds like a good idea to copy the midi buffer out and start entering your own data. That way you have full control and you don’t have to think about, if it is something you added or if it was in already.

No problem, I can only learn from these discussions… I hope my thinking is correct though, nobody knows it all.


#11

If you are correct, then just reading the buffer doesn’t clear it, so the events (if uncleared) would be send straight back to the host. That doesn’t sound right.

So if I would create an empty instrument project and simply not touch the midiBuffer, then all those events would be send straight back to the host? Maybe @jules can chime in?


#12

When the buffer is passed into processBlock, it contains the midi events that are being sent to your plugin. Whatever you leave in there gets sent as the output from your plugin, so if you don’t touch it, they get passed through.


#13

Can you elaborate? Passed through to who? So If my process function is completely empty, then what happens?

Do the events get sent back to the host (where they actually already come from)? Is that ever what a developer would expect to happen?

Does using “getNextEvent” from the MidiBuffer::Iterator remove the event from the buffer?

Do I have to call “midiMessages.clear ()” after to prevent the events being sent to the host?

What I’m trying to do:

  1. Read the events so my synth can play them.
  2. Add my own events when the user turns certain dials, as MIDI-CC, so the user can record them.

What is the correct approach here?

  1. Copy the events via a while-loop into my own buffer.
  2. midiEvents.clear ()
  3. Add my own events before returning.

Is that correct?


#14

The host records the midi that is coming in from the midi input, i.e. the outer world. This is sent into your track, passing through all effects.

  1. you get the same input if you are playing or playing back previously recorded midi. Otherwise the user wouldn’t be able to listen to the track having a different instrument selected in your synth, or even replacing your synth with a different one.
  2. afaik you cannot add anything to the recording, unless the host supports that. Sometimes there is a midi merge mode or similar, but again, that is not coming from your plugin.
  3. it is up to you, if you want your synth to consume the midi data, i.e. effectively clearing each buffer in processBlock(), or if you want to leave it. There might be the use case, that you want to have multiple instruments on that track, that all play the same thing.

To have this kind of overdub functionality, the user has to route the output of that track to a new midi channel and record there, I’m afraid. And if that’s even possible depends on which host we are talking about. Routing is the thing where there is the most difference between the hosts.


#15

I think there is some serious misunderstanding going on here.

What @daniel describes is simply impossible with VST2. You can’t modify the midi-stream that plugins after yours receive. The midi-events you get from VST2 are read-only.

So given that what @daniel describes is technically impossible, I wonder how the JUCE implementation of the midi-stuff really works, because it certainly doesn’t work like @daniel describes.

In VST2 land, there is a function to that gets called with the midi-events. It’s called processEvents () and it’s a read-only function. You can’t add events or change events.

Also in VST2 land there is a simple “sendVstEventsToHost” function. It’s a misnomer, as it only accepts midi-events.

That’s all I want to do. I want to know how to send midi events to the host, so the host can record them. I don’t want to pass midi events to following plugins, I don’t want to merge anything, I want to receive midi for my own consumption and send midi for the host to record (if recording is enabled). I don’t care how or if the host treats that data. I don’t care if it ignores it, merges it or replaces existing data.


#16

Ok, I am curious of alternative answers. But my understanding so far is, that each plugin is like an insert on a console. You get the signal, you leave it for the next, or you intercept it.

And that’s the same for sample data or midi events.


#17

For sample-data = true
For midi-data = not true

I’ve implemented VST2, AU, RTAS and AAX myself before switching to JUCE and none of these interfaces allows for the modification of midi-events that a plugin receives. It’s a strict read-only buffer that gets passed to all plugins in that track.

Sending to the host was always a special function that needed to be called.


#18

juce_VST_Wrapper.cpp has the answers:

How it really works:

  1. Juce converts the VST2 midi-events into its own midiBuffer by copying the data
  2. Juce calls your “processBlock” function.
  3. After “processBlock” returns, Juce checks if the midiBuffer it passed still contains events. If yes, it creates a new VstEvent buffer and uses “sendVstEventsToHost” to send it back to the host.

So to make it clear for everyone who might have the same problems/questions:

  1. Consume the events.
  2. Call “midiBuffer.clear ()”
  3. Optionally add the events you want to send to the host.

Even if you don’t want to send anything, make sure to clear the midi buffer. Otherwise, the events you’ve just received will be sent back verbatim to the host. This is completely unnecessary and could even cause problems.