Problem in initiating AudioSampleBuffer

#1

I’m using AuduiSampleBuffer to get an audio from Kontakt using a midi file in my host plugin app. When I loop over the MidiBuffer, the first audio buffers are somehow mixed up, and it takes about 5 seconds to sound right:

    FileInputStream fileStream(filename);
        MidiFile Mfile;
        Mfile.readFrom(fileStream);
        Mfile.convertTimestampTicksToSeconds();
        MidiBuffer midiBuffer;
        midiBuffer.clear();
        int totalTime;
        for (int t = 0; t < Mfile.getNumTracks(); t++) {
            const MidiMessageSequence* track = Mfile.getTrack(t);
            for (int i = 0; i < track->getNumEvents(); i++) {
                MidiMessage& m = track->getEventPointer(i)->message;
                std::cout<<m.getDescription ()<<" time:" << m.getTimeStamp() << "\n";
                int sampleOffset = (int)(sampleRate * m.getTimeStamp());
                midiBuffer.addEvent(m, sampleOffset);
                totalTime = m.getTimeStamp();
            }
        }
        int numberOfBuffers = int (std::ceil ((totalTime+0.5) * sampleRate / bufferSize));
        // Data structure to hold multi-channel audio data.
        AudioSampleBuffer audioBuffer (plugin->getTotalNumOutputChannels(),
                                       bufferSize);
        plugin->prepareToPlay (sampleRate, bufferSize);
        File outputFile = File(wavfile);
        WavAudioFormat format;
        std::unique_ptr<AudioFormatWriter> writer;
        writer.reset (format.createWriterFor (new FileOutputStream (outputFile),
                                              44100,
                                              audioBuffer.getNumChannels(),
                                              16,
                                              {},
                                              0));
      
        for (int i = 0; i < numberOfBuffers; ++i)
        {
            plugin->processBlock (audioBuffer, midiBuffer);
            if (writer != nullptr)
                writer->writeFromAudioSampleBuffer (audioBuffer, 0, audioBuffer.getNumSamples());
        }
        writer.reset(); 

Meanwhile, in order to get over the problem I added another midi-audio loop without writing the audio buffer to the file. Just to make it ready for the next loop…

for (int i = 0; i < numberOfBuffers; ++i)
{
    plugin->processBlock (audioBuffer, midiBuffer);
}
for (int i = 0; i < numberOfBuffers; ++i)
{
    plugin->processBlock (audioBuffer, midiBuffer2);
    if (writer != nullptr)
        writer->writeFromAudioSampleBuffer (audioBuffer, audioBuffer.getNumSamples());
}
writer.reset(); 

I believe it is related to initiating the audio buffer or the plugin. The midi file seems to be all right. Any ideas? Thanks!

0 Likes

#2

A shot in the dark, but if you are running in a loop without an audio driver, the processBlock calls are sent as fast as possible. Plugins, that are designed for live playback, will avoid blocking and return as fast as possible, to avoid stalling the audio thread.

Your host can try to call AudioProcessor::setNonRealtime (true) and if you are lucky, the plugin will switch to a blocking mode, not skipping audio blocks if it is not ready yet.

This helped me rendering offline, maybe it solves your problem too.

Good luck

0 Likes

#3

Maybe sending the MIDI messages into the plugin gets messed up because you are sending the whole MidiBuffer at once and multiple times…As far as I’ve understood, the buffer must contain only the MIDI messages that happen during the current audio buffer to be processed. (Or is your audio buffer actually so long that all the MIDI messages fit into that duration…?)

0 Likes

#4

Unfortunately setNoneRealtime(true) doesn’t help. Any other idea?

0 Likes

#5

Actually I read the midi file and want to write it to a wav file as fast as I can. I don’t want to play the midi file, so waiting about 45 seconds (the midi playing time) to produce a wav file doesn’t make sense. The fact that after 5 seconds the audio is good can tell that it is a problem of initialization (that what I think…). I changed the audio buffer length from 64 to 5120 (now it is 512), but there is no effect.

0 Likes

#6

I am not saying you should wait for anything. Just saying it may be that you are sending the wrong MIDI messages into the plugin with the processBlock calls.

0 Likes

#7

It still feels to me, like your Kontakt plugin takes 5 seconds to load its sounds, since you say, it sounds fine after 5 seconds. Just for research you could add a sleep before you start rendering.

@Xenakios idea was, that some plugins clear the whole midi buffer. Usually you are supposed only to send midi events, that happen during that AudioBuffer, since the timestamps of the MidiBuffer are supposed to be relative to the AudioBuffer in that processBlock() call.

The rendering would look like:

  • startSample = 0
  • loop:
  • add midi events from startSample to startSample + getNumSamples() to MidiBuffer
  • the timestamp is relative to the block, so use
    timestamp = m.getTimeStamp() * sampleRate - startSample
  • call processBlock
  • startSample += getNumSamples()

That’s the standard procedure, even though it doesn’t really explain your observation, that it sounds correct after a few seconds…

0 Likes

#8

Yeah, that throws me off a bit. If the rendering really is fine after some time has passed in the output, it would seem to be some kind of a problem with how Kontakt handles offline rendering. Maybe for testing purposes could try a sleep call of a couple of seconds after prepareToPlay to let the plugin initialize its internal state, if it happens to be doing that asynchronously…? But setNonRealTime(true) should probably handle those kinds of situations anyway…

0 Likes

#9

Research results:
Sleep(10) solves the problem (but who wants to wait 10 seconds?)
The second loop I added adds only 4 seconds, so this is a better patch.
I wonder if there any other elegant solution which takes less time to get audio from Kontakt…

0 Likes

#10

It is a matter of the plugin. Seems it doesn’t supply any info, when the samples are ready loaded.
Maybe get in touch with the developers of NI, since setting to non realtime should make the Kontakt plugin only render, once the samples have been loaded.

0 Likes

#11

Well, there’s really no proper way to pass that info between the plugin and the host anyway. (Unless one doesn’t consider hacks like having a parameter in the plugin that would work as a flag to tell if the plugin is ready to play.)

0 Likes