Inconsistencies when rendering an edit after inserting mp3 files

I’m inserting audio files into an edit’s audio track via someAudioTrack->insertWaveAudioClip(clipname, file, pos, false). When I go to render the edit immediately after inserting content into it, the behavior is different depending on whether the audio file I just inserted into the edit is a .wav file or a .mp3 file:

  • If the newly inserted file is a .wav file, rendering the edit works as expected
  • If the newly inserted file is a .mp3 file, then the contents of that file are not included in the rendered output.

Looking at the tracktion code, I can see that te::AudioClipBase derives from juce::Timer, and somewhere during playback, that timer callback adds a Render job which asynchronously creates a proxy .wav file in the edit’s temp directory.

Is there a way flush the proxy render jobs so that I can be sure that .mp3 files in an edit will be included when that edit is rendered?

If not, is there any way of firing a callback when all the proxy files are ready? I thought I would be able to use one of the following to detect the state of the proxy generation, but these seem to update asyncronously, and I haven’t figured out a reliable way to use them to after inserting an .mp3 file.

  • AudioClipBase::beginRenderingNewProxyIfNeeded()
  • edit->engine.getAudioFileManager().proxyGenerator.isProxyBeingGenerated(clip->getPlaybackFile())
  • AudioClipBase::needsRender()
  • AudioClipBase::getPlaybackFile()->getInfo().needsCachedProxy()

My application does not have a GUI, so there aren’t any natural pauses built into the operation. If there’s no other way, I can just wait approximately 1000-1500 milliseconds before rendering, but that makes things feel very sluggish, especially when rendering many edits.

Is there something I’m missing? Thank you for any advice or guidance!

The export/render should pause and wait for the proxy file to be completed.
The way this works is that the render graph is created with the proxy audio file AudioClipBase::getPlaybackFile().

Then in the old audio graph WaveAudioNode::isReadyToRender(), will return false until the proxy has rendered. In the new graph in NodeRenderContext::renderNextBlock you’ll see a lambda is called that is leafNodesReady. This basically waits until WaveNode::isReadyToProcess() return true which should only be when the proxy file has finished writing and has a valid sample rate.

Is this not happening for you?

Thank you, @dave96. I am seeing the offline edit render begin (and finish) before the .mp3 proxy rendering begins. When I play the edit, I can hear the inserted mp3 file (once the proxy rendering completes). However if I begin an offline render after inserting the mp3, the the contents of the mp3 are missing from the exported audio.

I put a breakpoint in WaveAudioNode::isReadyToRender() to see when it was getting called in relation to the edit rendering and the proxy rendering – however, that breakpoint isn’t triggered when I play the edit or when I render an edit. Could that mean that the renderer is not checking if the WaveAudioNodes are ready or not before the rendering process begins?

That’s odd. In tracktion_Renderer.cpp in the RenderContext class there’s this line:

        // wait for any nodes to render their sources or proxies
        while (! (node->isReadyToRender() || owner.shouldExit()))
            return false;

That should go through the graph and call isReadyToRender on all subnodes. Can you step through that and see if it’s going in to your WaveAudioNode or if it’s jumping out early?

Thanks @dave96 I stepped the isReadyToRender() code, and found a custom rack instance on the track that contained the mp3 file. The RackType::isReadyToRender() call is returning true even before the mp3 proxy is rendered. Here’s what the rack looks like when opened in waveform:

I tried removing the rack instance, but that revealed another problem. Now WaveAudioNode::isReadyToRender() is getting called, but it returns false indefinitely via line 96. This causes the rendering job to hang in an infinite loop on the message thread. It seems like the proxy generator job cannot start (or was never scheduled?) while the rendering job is spinning.

I am using a March 2nd commit from the tracktion_graph branch.

[edit] So I guess that are two questions now:

  • Is it the expected behavior that the RackType::isReadyToRender() returns true even when inputs to the rack are not ready to render?
  • Is it also the expected behavior that calling te::Renderer::renderToFile on the message thread should block proxy files from being created?

This looks like a bug when using the old engine. This shouldn’t happen with the new engine as the leaf nodes are checked to see if they’re ready.

Can you switch to the develop branch and enable set ENABLE_EXPERIMENTAL_TRACKTION_GRAPH=1 then call EditPlaybackContext::enableExperimentalGraphProcessing (true), that should switch to using the new engine and solve the problem.

Okay, I tried this, and set a breakpoint in WaveNode::isReadyToProcess. When I go to run an offline render, the breakpoint is triggered, but the function is still returning true - this time line 81. This happens even during an offline render. Could be be that the isOfflineRender flag is not getting set correctly?

Hmm, I would have thought that logic is correct but maybe because the proxy is a wav file, it’s actually writing the header first and then appending blocks, updating the header later on?

I was going to suggest seeing if we could check the lengthInSamples and avoid creating a reader if a file has 0 length but I think that case is already handled too:

MemoryMappedAudioFormatReader* WavAudioFormat::createMemoryMappedReader (FileInputStream* fin)
{
    if (fin != nullptr)
    {
        WavAudioFormatReader reader (fin);

        if (reader.lengthInSamples > 0)
            return new MemoryMappedWavReader (fin->getFile(), reader);
    }

    return nullptr;
}

So I tried replicating this by loading a 2hr MP3 in to Waveform and trying to render it whilst the proxy is still being created. However, this just waits until the proxy is finished because the audioFileSampleRate is 0 before the proxy is created (this is because the MemoryMappedWavReader above can’t be created until the file has finished rendering).

I’m not sure why this would be different in your use case?
Where exactly is isReadyToProcess() returning true? Are you saying that isOfflineRender == false even when you’re rendering?

Oh hang on. I’ve misunderstood what’s going on. I think I see the problem now.

Can you hack AudioClipBase to make the Timer base class and timerCallback methods public?
Then just before you render, call timerCallback on all your AudioClipBases?

If that does what you need, I’ll think of a sensible way to add that functionality before a render operation.

Okay, I tried this out. timerCallback invokes WaveAudioClip::needsRender() (via AudioClipBase::shouldAttemptRender). However, needsRender() returns false and the proxy is still not rendered. Looking at the code, it looks the conditions on line 96 are not met. Should that line check if the file is compressed?

Thank you for looking at this!

No… needsRender is correctly returning false. It’s confusing but that’s for source file rendering like EditClips, ClipEffects or reversing. It should jump down to line 2600:

const AudioFile newProxy (getPlaybackFile());

const bool proxyChanged = lastProxy != newProxy;

and proxyChanged should be true which should end up calling new ProxyGeneratorJob?

Okay, calling timerCallback solves the problem - as long as I don’t have a rack in the audio graph.

It appears that even when using the tracktion_graph module alongside tracktion_engine with the ENABLE_EXPERIMENTAL_TRACKTION_GRAPH=1 preprocessor definition, PluginAudioNode::isReadyToRender returns true without checking the inputs. Is that the expected behavior? If not, perhaps I did not correctly enable tracktion_graph. Is there a way I can confirm that it is being used?

I got some strange results where mp3 source content processed by a rack instance can be heard only towards the end of an audio file that was rendered offline.

To me it suggests that when rendering offline, isReadyToRender for rack types behaves like it does for live playback: Playback starts immediately even if proxies are not already generated. Once the proxy content is rendered, only then it is audible. I’m not sure if that’s what is happening or not, but it would explain the strange results I just got.

Hmm, if you’re getting any of the xxxAudioNode classes being called it doesn’t sound like you have the new engine enabled properly? (All of the new classed are simply xxxNode.

In the new engine, it’s the leaf Nodes i.e. the WaveNodes that are checked to see if they can render before starting.

Are you sure you’re calling EditPlaybackContext::enableExperimentalGraphProcessing (true) before you open any Edit?

BTW, I know this is confusing atm, we’re in the final optimisation and stability stages of the new engine and then we’ll remove the old one so there’s no ambiguity.

Hmm I did call EditPlaybackContext::enableExperimentalGraphProcessing (true) before loading edit, and it looks to me like it is correctly enabled. Below is the call stack that results in PluginAudioNode::isReadyToRender. Should this not happen?

It seems like it is the combination of a .mp3 file and a rack that causes the problem. A .wav file played through a rack works fine. An mp3 file that plays through a rack outputs silence, and (IIRC) an mp3 played without a rack works fine (assuming timerCallback was called).

However, looking closely I can see that the offline render is now (correctly) waiting to begin until the proxy render is complete – so it would appear that the rack is correctly waiting for its inputs, and I am not sure why the mp3 content is excluded from the output of the offline render.

Ok, I see the problem Renderer::renderToFile will only use the old engine.
I’m really tied up in something else at the moment but I’ll sort this out once I’ve finished some other changes with the engine.

1 Like

Hi Dave, did you ever get a chance to update renderToFile so that it can use the new engine?

Not yet I’m afraid. I’ll soon be embarking on removing the old engine so I can find any places that reference it. I’ll do so on a branch so you can follow the progress.

2 Likes