(Solved) Is it possible to display a progress bar while loading an edit?

When loading an edit, UI becomes unresponsive for the duration of the load operation. The edit is loaded on the message thread, which as far as I can tell, is required by TE.

I noticed in the Edit::Options struct, there is an optional Edit::LoadContext member that can be assigned to receive loading progress updates as well as the ability to cancel loading, and a completion flag.

But, I’m a little lost as to how to properly use this from the UI. TE hits the message thread really hard when it is loading an edit, and since it seems all UI repainting is done on the message thread as well, the ability to sneak in some repaints to animate a progress bar doesn’t seem possible until the edit is finished loading.

I’ve tried this with ThreadWithProgressWindow using launchThread and then loading the edit. The window opens, but the LoadContext.progress value within the ThreadWithProgressWindow thread reports 0 the whole time until all of the sudden it is magically 1 and then closes (and yes I’m loading the edit with a LoadContext and passing its reference to the window).

Given the edit and UI’s reliance on the message thread, I don’t see any easy way of rendering progress animation, which leaves me a little confused about the purpose of the LoadContext. That said, I’m not super savvy with TE and the intricacies of the message thread yet, so any insights would be appreciated. Thanks!

1 Like

Can you share the code of your ThreadWithProgressWindow subclass here?

https://docs.juce.com/master/classThreadWithProgressWindow.html#details

(Caveat: I use JUCE but not Tracktion Engine, so I might not understand if there’s something TE-specific happening here.)

OK, so I have a wrapper class Player that encapsulates the Tracktion Engine classes. All interactions with TE are wrapped including loading an edit, and it also has a te::Edit::LoadContext member named loadContext.

te::Engine engine { ProjectInfo::projectName, std::make_unique<ExtendedUIBehaviour>(), nullptr };
std::unique_ptr<te::Edit> edit = nullptr;
te::Edit::LoadContext loadContext;

It has a simple getter to grab a reference to the LoadContext member:

te::Edit::LoadContext& getLoadContext() { return loadContext; }

The Player class has a public loadEdit(String, String) method that eventually calls this private load method internally (which is identical to the te::loadEditFromFile() method, except it resets/adds the LoadContext to the options):

std::unique_ptr<te::Edit> Player::loadEditFromFile(te::Engine& engine, const juce::File& editFile)
{
	loadContext.progress = 0;
	loadContext.shouldExit = false;
	loadContext.completed = false;

	auto editState = te::loadEditFromFile(engine, editFile, {});
	auto id = te::ProjectItemID::fromProperty(editState, te::IDs::projectID);

	if (!id.isValid())
		id = te::ProjectItemID::createNewID(0);

	te::Edit::Options options =
	{
		engine,
		editState,
		id,

		te::Edit::forEditing,
		&loadContext,
		te::Edit::getDefaultNumUndoLevels(),

		[editFile] { return editFile; },
		{}
	};

	return std::make_unique<te::Edit>(options);
}

Now in the subclass of ThreadWithProgressWindow:

class LoadEditProgressWindow  : public juce::ThreadWithProgressWindow
{
public:
    LoadEditProgressWindow(Player& p) : ThreadWithProgressWindow("Loading...", true, true), player(p)
    {
    }

    ~LoadEditProgressWindow() override
    {
    }

    void run() override
    {
        while (!threadShouldExit())
        {
            auto& loadContext = player.getLoadContext();
            auto progress = loadContext.progress.load();
            auto completed = loadContext.completed.load();
            setProgress(progress);
            //DBG("Progress: " + String(progress));
            if (completed) break;
            sleep(10);
        }
    }

private:
    Player& player;
    //==============================================================================
    JUCE_DECLARE_NON_COPYABLE_WITH_LEAK_DETECTOR (LoadEditProgressWindow)
};

Finally, in a button handler I am opening the progress window and then starting the load of the edit:

LoadEditProgressWindow loadProgressWindow { player };
loadProgressWindow.launchThread();

// Load the edit into the player
player.loadEdit(editFilePath, projectFilePath);

If I debug the output of progress from within the window’s run() method I basically get a bunch of 0’s and then finally 1 and it closes because completed is true when progress is 1.


But as I mentioned before, I think the actual issue here is that all JUCE UI becomes unresponsive during the load of te::Edit because loading happens on the message thread, and when it is loading, the message thread is busy with creating the corresponding audio graph, and can’t handle other messages such as repaint() calls.

I tried loading an edit on another thread but hit an error message that seemed to indicate it needs to be loaded from the message thread. Maybe there is a way to load an edit on a different thread to keep UI responsive, but I haven’t found it yet.

What error were you hitting? You can load Edits on a background thread, that’s what we do in Waveform.

Whilst most of the Edit can be loaded on a background thread, the time consuming bits like loading plugins and setting plugin state have to happen on the message thread, that’s why they’re wrapped in callBlocking calls.

After more poking the error I was getting was due to the fact that I was doing some extra things after loading the edit, such as getting references to plugins and tracks, setting the transport position, etc. which I was calling in the background thread after the edit had loaded, which caused an error about the message thread being locked. I made the background thread loader class inherit from AsyncUpdater and used that to asynchronously invoke the finalization bits back on the message thread post-load, and that made everything happy.

However, I’m still basically seeing the issue of the progress window thread just reporting progress as 0, and then suddenly 1. I finally looked into the code and found this in Edit::initialise():

...

if (loadContext != nullptr)
    loadContext->progress = 0.0f;

    treeWatcher = std::make_unique<TreeWatcher> (*this, state);

    isLoadInProgress = true;
    tempDirectory = juce::File();

    if (! state.hasProperty (IDs::creationTime))
        state.setProperty (IDs::creationTime, juce::Time::getCurrentTime().toMilliseconds(), nullptr);

    lastSignificantChange.referTo (state, IDs::lastSignificantChange, nullptr,
                                   juce::String::toHexString (juce::Time::getCurrentTime().toMilliseconds()));

    globalMacros = std::make_unique<GlobalMacros> (*this);
    initialiseTempoAndPitch();
    initialiseTransport();
    initialiseVideo();
    initialiseAutomap();
    initialiseClickTrack();
    initialiseMetadata();
    initialiseMasterVolume();
    initialiseRacks();
    initialiseMasterPlugins();
    initialiseAuxBusses();
    initialiseAudioDevices();
    loadTracks();

    if (loadContext != nullptr)
        loadContext->progress = 1.0f;

...

I searched for all references to loadContext in te::Edit’s source and the only relevant bit I found is the snippet above. Maybe the intention was to incrementally update loadContext.progress after or inside of each of those initialise methods, but nothing actually does. From what I can tell the code literally just sets it to 0, and then 1 when everything is finished, which supports the behavior I’m seeing. It looks like a feature that was intended to be implemented, but never fully was?


Update: So I actually just went in and added lines to manually update loadContext->progress starting after initialiseTempoAndPitch(); and finishing with initialiseAudioDevices(); - basically after each initialise method I added a line to increment the progress. Adding those lines, combined with loading the edit on a background thread, and using AsyncUpdater to finish some post-load custom bits on the message thread has finally done the trick with an animating progress bar!

I’m not sure how loadContext.progress was meant to be implemented, but it turns out it just hasn’t been setup properly.

Yeah, it used to work differently but the structure changed which makes it harder to accurately report load progress. The problem is that almost all of the time loading is spent in plugin creation and state loading so really you want to update progress at least after every track load, but that doesn’t happen in the Edit now, it happens inside nested TrackLists and I was trying to think of a way to nicely update the progress here.

For this reason, in Waveform we just show a spinning wait animation.
In the way you’ve done it, do you just get a big jump to the start of loadTracks, a big pause and then another big jump to 1.0?

In testing it, the edits I’m loading aren’t necessarily super complex, and in some cases it gets to 10% and then jumps to 90% and is finished, like you are predicting. But honestly, not always. I definitely get a full range of progress values, lets say up to 58% or so, and then there is usually a big jump for the final bits. I would say it’s fit for purpose though, it definitely gives feedback of the progress of loading. Sometimes I see progressive values all the way up to 100%, it just varies.

That said, I’ve had to alter the te::Edit’s code to make it work, and I think I will just stick with the vanilla code, rather than a custom patch, and just put up a spinner as you mentioned. Mostly I wanted to be able to put up some sort of loading indicator that didn’t freeze during load.

My original loading method that did everything from the message thread notified UI that loading was done via an ActionBroadcaster call, but when I moved it to a background thread I failed to section that bit out via AsyncUpdater, so when UI began to respond, that’s when I got the error about the message thread being locked, and at first I presumed it was TE complaining it wasn’t being loaded on the message thread; just my ignorance as a newbie working with TE.

As always, I really appreciate your help and responsiveness. Thanks!