Loading video asynchronously on Windows

[cpr2323]

I would like to load a video asynchronously, so as not to block the message thread, but on Windows VideoComponent::loadAsync just calls VideoComponent::load. I assume I need to implement my own thread to load in the background.

a) I wanted to verify that I need write some ifdefed code for windows to use a thread, but can rely on loadAsync for OSX, iOS, and Android?

b) wouldn’t it make sense for JUCE VideoComponent::loadAsync to use it’s own thread, so that the function actually operates the same on all platforms?

[cpr2323]

ACK! ffs… VideoComponent::load calls resized() when it’s done… which of course, can’t be called in a thread… and, it means VideoComponent::load operates differently than VideoComponent::loadAsync, which does not call resized()… JUCE team, can you assist with this? either implement a proper loadAsync() for Windows, or allow us a method to call load() from a thread?

[ed95]

Yes, loadAsync() should work fine for iOS, macOS and Android. We probably won’t have time to look at implementing a proper loadAsync() on Windows in the near future, but would removing the call to resized() work for your use case for the time being?

[cpr2323]

That would totally work! I have already written the code to do the load on it’s own thread, and discovered the call to resized() when testing it the first time. Thanks.

[ed95]

OK, I’ll put something together.

[ed95]

Don't call resized() in VideoComponent::loadAsync() on platforms which don't have a proper implementation to allow the method to be called from user threads

[cpr2323]

Testing it now. Video is not displaying, but I do hear audio. Here is my code:

void VideoPlaybackComponent::run ()
{
    videoComponent.loadAsync (URL (File (videoFileName)), [this] (const URL&, Result result)
                              {
                                  MessageManager::callAsync ([this, result] () { resized (); videoLoadComplete (result.wasOk ()); });
                              });
}

[cpr2323]

I’ll work on debugging it a little later today.

[ed95]

Hmm, I’m not seeing any issues using loadAsync(). It could be that the VideoComponent wasn’t being resized correctly after that change, so I’ve pushed 404455b which will start the resize timer after loading if done asynchronously. Can you see if that fixes your issue? If not then could you post a simple example which reproduces the issue and I’ll look into it.

[cpr2323]

I am still having issues. I verified using load from the message thread is still working. I know you asked for a simple example, and I hope the class I’ve implemented is considered simple enough, otherwise let me know, and I will reduce it even more. As well, here is a video showing what I am experiencing.
Loading Video from Thread

VideoPlaybackComponent.h

#pragma once

#include "../JuceLibraryCode/JuceHeader.h"

#if JUCE_WINDOWS
#define ENABLE_THREADED_LOAD 1
#else
#define ENABLE_THREADED_LOAD 0
#endif // JUCE_WINDWOS
class VideoPlaybackComponent : public Component,
                               private Timer
#if ENABLE_THREADED_LOAD
                              , private Thread
#endif // ENABLE_THREADED_LOAD
{
public:
    VideoPlaybackComponent ();
    ~VideoPlaybackComponent ();

    void setup (String coverImageFileName, String videoFileName);

private:
    String coverImageFileName;
    String videoFileName;
    ImageComponent coverImageComponent;
    VideoComponent videoComponent { false };
    Slider videoPositionSlider { Slider::SliderStyle::LinearHorizontal, Slider::TextEntryBoxPosition::NoTextBox };

    std::atomic<bool> videoLoaded { false };
    std::atomic<bool> videoLoading { false };
    std::atomic<double> videoLength { 0.0 };
    void loadCoverImage ();
    void loadVideoAsync ();
    void togglePlay ();
    void setPlaybackPosition (double position);
    void videoLoadComplete (bool success);

    void timerCallback () override;
    void mouseUp (const MouseEvent& e) override;
    void resized () override;
    void paint (Graphics& g) override;
#if ENABLE_THREADED_LOAD
    void run () override;
#endif // ENABLE_THREADED_LOAD
};

VideoPlaybackComponent.cpp

#include "VideoPlaybackComponent.h"

VideoPlaybackComponent::VideoPlaybackComponent ()
#if ENABLE_THREADED_LOAD
    : Thread {"VideoPlaybackComponent"}
#endif // ENABLE_THREADED_LOAD
{
    setOpaque (true);

    // add video component, but don't make it visible, since we will be displaying the 'cover image' until the video is loaded
    videoComponent.setInterceptsMouseClicks (false, false);
    addChildComponent (videoComponent);

    coverImageComponent.setInterceptsMouseClicks (false, false);
    addAndMakeVisible (coverImageComponent);

    videoPositionSlider.setEnabled (false);
    videoPositionSlider.onValueChange = [this] () { setPlaybackPosition (videoPositionSlider.getValue ()); };
    addAndMakeVisible (videoPositionSlider);
}

VideoPlaybackComponent::~VideoPlaybackComponent ()
{
#if ENABLE_THREADED_LOAD
    if (isThreadRunning ())
        stopThread (1000);
#endif // ENABLE_THREADED_LOAD
}

void VideoPlaybackComponent::setup (String cifn, String vfn)
{
    coverImageFileName = cifn;
    videoFileName = vfn;

    loadCoverImage ();
    loadVideoAsync ();
}

void VideoPlaybackComponent::loadCoverImage ()
{
    File imageFile { coverImageFileName };
    FileInputStream imageFileStream { imageFile };
    if (imageFileStream.openedOk ())
    {
        const auto imageFileFormat { ImageFileFormat::findImageFormatForStream (imageFileStream) };
        const auto coverImage = imageFileFormat->decodeImage (imageFileStream);
        coverImageComponent.setImage (coverImage);
    }
}

void VideoPlaybackComponent::togglePlay ()
{
    if (videoComponent.isPlaying ())
    {
        stopTimer ();
        videoComponent.stop ();
    }
    else
    {
        startTimer (250);
        videoComponent.play ();
    }
}

void VideoPlaybackComponent::timerCallback ()
{
    if (videoLoaded.load ())
        videoPositionSlider.setValue (videoComponent.getPlayPosition (), NotificationType::dontSendNotification);
}

void VideoPlaybackComponent::mouseUp (const MouseEvent&)
{
    togglePlay ();
}

void VideoPlaybackComponent::paint (Graphics& g)
{
    g.fillAll (Colours::white);
}

void VideoPlaybackComponent::resized ()
{
    auto bounds { getLocalBounds () };
    videoPositionSlider.setBounds (bounds.removeFromBottom (getHeight () / 20));
    coverImageComponent.setBounds (bounds);
    videoComponent.setBounds (bounds);
}

void VideoPlaybackComponent::setPlaybackPosition (double position)
{
    if (videoLoaded.load ())
        videoComponent.setPlayPosition (position);
}

void VideoPlaybackComponent::videoLoadComplete (bool success)
{
    jassert (success);
    if (success)
    {
        coverImageComponent.setVisible (false);
        videoComponent.setVisible (true);
        videoPositionSlider.setRange (0.0, videoComponent.getVideoDuration ());
        videoPositionSlider.setEnabled (true);
        videoLoaded.store (success);
    }
    videoLoading.store (false);
}

void VideoPlaybackComponent::loadVideoAsync ()
{
    videoLoaded.store (false);
    videoLoading.store (true);
#if 1
#if JUCE_MAC || JUCE_IOS || JUCE_ANDROID || !ENABLE_THREADED_LOAD
    videoComponent.loadAsync (URL (File (videoFileName)), [this] (const URL&, Result result)
                              {
                                  videoLoadComplete (result.wasOk ());
                              });
#else
    startThread ();
#endif // JUCE_MAC || JUCE_IOS || JUCE_ANDROID
#else
    videoComponent.load (File (videoFileName));
    videoLoadComplete (true);
#endif // 0
}

#if ENABLE_THREADED_LOAD
void VideoPlaybackComponent::run ()
{
    videoComponent.loadAsync (URL (File (videoFileName)), [this] (const URL&, Result result)
                              {
                                  MessageManager::callAsync ([this, result] ()
                                                             {
                                                                 videoLoadComplete (result.wasOk ());
                                                             });
                              });
}
#endif // ENABLE_THREADED_LOAD

[ed95]

Thanks for the example, I’ll take a look into this next week.

[ed95]

From looking at the implementation of the Windows VideoComponent, I don’t think it’s quite as easy as just calling the loadAsync() method on a thread. There’s a lot of code in the internal loading which needs to be called on the message thread for things like creating the native HWND which displays the video. Unfortunately I can’t see an easy way of pulling this out without basically rewriting most of the class, I can add this to the backlog but I’m afraid it’s not particularly high priority.

[cpr2323]

No worries. Thanks for the effort.