Send midi messages to VST from host to get audio back saving to wav file

Hello. I’m trying to implement a command line VST host that gets a midi file and connect Kontakt to save it as a wav file (using existing Kontakt patch).
I succeeded to get Kontakt parameters using :
plugin->getParameterName(i)
after initiating the plugin “Kontakt.component”.
However when trying to send a midi message and storing the audio in a wav file I got a very short sound, no matter which note I write. This is the code:

void RenderKontakt::getWave (const int midiNote,
const uint8 midiVelocity,
const double noteLength)
{
MidiMessage onMessage = MidiMessage::noteOn (1,
midiNote,
midiVelocity);
MidiBuffer midiNoteBuffer;
double startTime = (Time::getMillisecondCounterHiRes() * 0.001);
AudioSampleBuffer audioBuffer (plugin->getTotalNumOutputChannels(),
bufferSize);
auto message = MidiMessage::noteOn (2, midiNote, (uint8) midiVelocity);
message.setTimeStamp (Time::getMillisecondCounterHiRes() * 0.001 - startTime);
auto timestamp = message.getTimeStamp();
auto sampleNumber = (int) (timestamp * sampleRate);
midiNoteBuffer.addEvent (message, sampleNumber);

plugin->prepareToPlay (sampleRate, bufferSize);
plugin->processBlock (audioBuffer, midiNoteBuffer);

File outputFile = File("~/Development/Juce/Midi2KontaktTest/Kontakt.wav");

WavAudioFormat format;
std::unique_ptr<AudioFormatWriter> writer;
writer.reset (format.createWriterFor (new FileOutputStream (outputFile),
                                      48000.0,
                                      audioBuffer.getNumChannels(),
                                      24,
                                      {},
                                      0));
if (writer != nullptr)
    writer->writeFromAudioSampleBuffer (audioBuffer, 0, audioBuffer.getNumSamples());
writer.reset();

}

Any idea how to get the proper audio from the Kontakt VST or what I’m doing wrong?
Thanks.

How big is audioBuffer?
You’re going to have to keep calling processBlock() until the note should end.
And maybe Kontakt will stop playing if you call it faster than it streams the audio from disk (if not set to play the samples from memory, or to render offline)

Thanks! Indeed I have to call processBlock() in a loop, using “RenderMan” example ( bufferSize = 256, renderLength=3.0):
in the h file:
std::vector< float > processedMonoAudioPreview;

in cpp:
{
int numberOfBuffers = int (std::ceil (renderLength * sampleRate / bufferSize));
for (int i = 0; i < numberOfBuffers; ++i)
{
// Turn Midi to audio via the vst.
plugin->processBlock (audioBuffer, midiNoteBuffer);
readptrs = audioBuffer.getArrayOfReadPointers();
for (int j = 0; j < data.getNumSamples(); ++j)
{
// Mono the frame.
int channel = 0;
auto currentFrame = readptrs[channel][j];
const int numberChannels = data.getNumChannels();
while (++channel < numberChannels)
currentFrame += readptrs[channel][i];
currentFrame /= numberChannels;
// Save the audio for playback and plotting!
processedMonoAudioPreview.push_back (currentFrame);
}
}
}

Looking at processedMonoAudioPreview I can see indeed about 32,000 floats numbers between 0 to 1 (should it be like that?).
Now I have some difficulty to write it to a wav file since I have a double array so I can’t use:
writer->writeFromAudioSampleBuffer (audioBuffer, 0, audioBuffer.getNumSamples());

Am I using the right approach? How can I save to wav file this double array? Should I use a very big AudioSampleBuffer instead? Any example? Thanks!

There should not be any reason to involve a std::vector<float> when everything else is JUCE based code that uses AudioBuffers.

You are right. That what I suspected. How do you think I should implement it? How to save many AudioBuffers and then write it all to a wave file ? Thanks!

Something like this :

int64_t outlen = 10 * 44100;
	int bufsize = 512;
	AudioBuffer<float> procbuf(2, bufsize);
	AudioFormatWriter* writer = nullptr; // obviously init this to a valid writer instead...
	AudioProcessor* plugin = nullptr; // of course needs to be a valid AudioProcessor/plugin instance
	int64_t outcounter = 0;
	MidiBuffer midimessages;
	while (outcounter < outlen)
	{
		plugin->processBlock(procbuf,midimessages);
		writer->writeFromAudioSampleBuffer(procbuf, 0, bufsize);
		outcounter += bufsize;
	}
	delete writer;

It’s obviously missing some things, particularly how you would fill the MidiBuffer going into the plugin. (It needs to contain just the MIDI messages for the current buffer to be processed, not all your MIDI data at once…) I haven’t worked with that stuff much myself, so I don’t have a proper solution for doing that.

In theory you might be able to render the whole file at once without the while loop, using a big enough AudioBuffer for the processing, but all plugins may not deal with that gracefully. (It would also increase your program’s memory usage during the rendering.)

Thank you very much! Now it works!
I succeeded to sent a note to Kontakt and get the audio into a wave file.
Now I have to implement a midi sequencer that reads a midi file and send the commands to MidiBuffer. Do you have any example for doing that? Thanks again!

You may find this interesting: it (among many other things) reads a midi file, and streams the midi out in a MidiBuffer. Github page has links to the site that explains the whole thing. https://github.com/tomto66/Topiary-Beatz

1 Like

Thank you all for your useful advises. Here is my final working code including reading the midi file, sending to plugin and store the audio in a wav file:

void RenderKontakt::loadMidi (const String& filename)
{
    
    FileInputStream fileStream(filename);
    MidiFile Mfile;
    Mfile.readFrom(fileStream);
    Mfile.convertTimestampTicksToSeconds();
    MidiBuffer midiBuffer;
    midiBuffer.clear();
    int totalTime;
    for (int t = 0; t < Mfile.getNumTracks(); t++) {
        const MidiMessageSequence* track = Mfile.getTrack(t);
        for (int i = 0; i < track->getNumEvents(); i++) {
            MidiMessage& m = track->getEventPointer(i)->message;
            //std::cout<<m.getDescription ()<<" time:" << m.getTimeStamp() << "\n";
            int sampleOffset = (int)(sampleRate * m.getTimeStamp());
            midiBuffer.addEvent(m, sampleOffset);
            totalTime = m.getTimeStamp();
        }
    }
    int numberOfBuffers = int (std::ceil ((totalTime+0.5) * sampleRate / bufferSize));
    // Data structure to hold multi-channel audio data.
    AudioSampleBuffer audioBuffer (plugin->getTotalNumOutputChannels(),
                                   bufferSize);
    plugin->prepareToPlay (sampleRate, bufferSize);
    File outputFile = File("/Users/didi/Development/Juce/Midi2KontaktTest/Kontakt.wav");
    WavAudioFormat format;
    std::unique_ptr<AudioFormatWriter> writer;
    writer.reset (format.createWriterFor (new FileOutputStream (outputFile),
                                          44100,
                                          audioBuffer.getNumChannels(),
                                          16,
                                          {},
                                          0));
    
    for (int i = 0; i < numberOfBuffers; ++i)
    {
        plugin->processBlock (audioBuffer, midiBuffer);
        if (writer != nullptr)
            writer->writeFromAudioSampleBuffer (audioBuffer, 0, audioBuffer.getNumSamples());
    }
    writer.reset();
    
}
1 Like