Creating threads for file export

Audio Plugins
1

I’m writing this plugin that at some point has recorded loops, stored in AudioSampleBuffers. I have built a save button, and when you click it and select a directory, the plugin will write all buffers with something in them out to .wav files in the selected directory. So far so good.

Now, I want the save button to become an export button that you can simply drag, a la Maschine. I have a feeling I need to build and implement some custom class for this that would inherit from Thread. I am trying with the following:

// FileExportThread.h
class FileExportThread : public Thread
{
public:
    void giveAudioSampleBuffer (AudioSampleBuffer* newBufferptr);
    void run() override;
    
    class Listener
    {
    public:
        virtual ~Listener() {}
        virtual void fileWritten (String fileName) = 0;
    };
    
private:
    AudioSampleBuffer buffer;
};

// FileExportThread.cpp
#include "FileExportThread.h"

void FileExportThread::giveAudioSampleBuffer (AudioSampleBuffer *newBufferptr)
{
    buffer = AudioSampleBuffer (*newBufferptr);
}

void FileExportThread::run()
{
    if (buffer.getNumSamples() == 0)
    {
        listeners.call (&fileWritten, "");
        exit (0);
    }
    
    WavAudioFormat waf;
    std::unique_ptr<AudioFormatWriter> afw (waf.createWriterFor (new FileOutputStream (*new File (
                                                                                                  File::getSpecialLocation (File::tempDirectory).getFullPathName() + "/filename.wav")
                                                                                       ), sampleRate, kChannels, 16, StringPairArray(), 0));
    
    afw->writeFromAudioSampleBuffer (buffer,
                                     0,
                                     buffer.getNumSamples());
    afw->flush();

    listeners.call (&fileWritten, "..filename.wav");
}

However, on both lines with listeners.call, XCode shows 'listeners' is a private member of 'juce::Thread'. Understandable. But I need to know from the main GUI thread of my plugin, when all 8 loops have been successfully written, to then call the DragAndDropContainer::performExternalDragDropOfFiles with them.

Am I better off using the Thread::launch()?

What is the supposed way to communicate from a juce::Thread back to the spawner that the task has been successfully executed (hopefully with result in a String)?

Is there a better, easier, or cleaner way to launch exactly 8 threads to start writing .wav files and get notified when (all) done?

2

I’d check out
https://docs.juce.com/master/classAudioFormatWriter_1_1ThreadedWriter.html

and
https://docs.juce.com/master/classThreadPool.html

to see if they might help.

Since the thread is a subclass, you can add a member which the ‘managing class’ can read to communicate from the thread to the manager. Listener callbacks are on the same thread, so they wouldn’t be my first thought here.

1 Like
3

First of all, thank you for the excellent tips and links!

I’m getting into the ThreadPool now. So kind of a separate issue, but could you tell me how to fix the following error?

exportThreadPool.addJob([]() -> juce::ThreadPoolJob::JobStatus {
   // ...
   return ThreadPoolJob::jobHasFinished;
});

I’m getting “Call to member function ‘addJob’ is ambiguous”, but I am unsure how I could further instruct the compiler into which of the two (link one and two) addJob functions to choose from.

4

Found it.
For anybody coming here after 2023, it had to be:

exportThreadPool.addJob(std::function<juce::ThreadPoolJob::JobStatus()>([]() -> juce::ThreadPoolJob::JobStatus {
   // ...
   return ThreadPoolJob::jobHasFinished;
}));

Found on StackOverflow

5

I want to post the solution I have found, for if somebody lands here in the future.

I have put all the logic around threads saving the buffers in the GUI thread (ie PluginEditor).

// PluginEditor.h
class PluginEditor : ...
{
...
private:
    bool                                waitingForFiles;
    StringArray                         exportFileNames;
    ThreadPool                          exportThreadPool;
}

where waitingForFiles is a flag that signals later to the timerCallback() that it can initiate the drag-and-drop. It’s initialised to false in the constructor. exportThreadPool is initialised with simply the amount of tracks I’m trying to save (amount of tracks is a constant in my plugin).

I have a button class ExportButton that derives from my own button component, but I think one could just as easily derive it from juce’s Button. You just need to override a few things: mouseDown, mouseUp, and add a listener class of its own.

// ExportButton.h
class ExportButton : public Button
{
                                ExportButton();
    
    void                        mouseDown (const MouseEvent& e) override;
    void                        mouseUp (const MouseEvent& e) override;
    
    void                        filesReady (StringArray& fileNames);
    
    class Listener
    {
    public:
        virtual ~Listener() {}
        virtual void dragStarted (ExportButton* button) = 0;
    };
    
    void                        addDragListener (Listener* newListener) { dragListeners.add (newListener); }
    
private:
    ListenerList<Listener>      dragListeners;
    
    JUCE_DECLARE_NON_COPYABLE_WITH_LEAK_DETECTOR (ExportButton)
};
typedef ExportButton::Listener ExportButtonListener;

It’s important to name the new listener list something different then listeners, and the add-listener too, to not interfere with the existing inherited and private members.

The cpp side is very very simple:

// ExportButton.cpp
...

void ExportButton::mouseDown (const MouseEvent& e)
{
    MouseCursor::showWaitCursor();
    dragListeners.call (&Listener::dragStarted, this);
}

void ExportButton::mouseUp (const MouseEvent& e)
{
    MouseCursor::hideWaitCursor();
}

The show/hide wait cursor does not yet seem to work in VST/AU world. So still trying to find a way to at least show it on the button. The important part here is calling the dragStarted event listeners on mouse down, so that the plugin editor can start the writing of files immediately.

// PluginEditor.cpp
PluginEditor::PluginEditor (ExampleAudioProcessor* ownerAudioProcessor)
{
    ...
    addAndMakeVisible (*btnExport);
    btnExport->setLabel ("EXPORT");
    btnExport->addDragListener (this);
    ...
}

...

void PluginEditor::dragStarted (ExportButton* button)
{
    if (button == btnExport.get())
    {
        // remove old files
        File tempDir (File::getSpecialLocation (File::tempDirectory));
        for (auto file : tempDir.findChildFiles (File::TypesOfFileToFind::findFiles, false, "*"))
            file.deleteFile();
        
        // in separate threads, write new files
        waitingForFiles = true;
        exportFileNames.clear();
        exportThreadPool.removeAllJobs (true, 2000);
        for (int i = 0; i < kTracks; i++)
            exportThreadPool.addJob(std::function<juce::ThreadPoolJob::JobStatus()>([this, i]() -> juce::ThreadPoolJob::JobStatus {
               auto fileName = this->trackGUIs[i].saveBuffer (File::getSpecialLocation (File::tempDirectory).getFullPathName());
               if (fileName != "")
                  exportFileNames.add (fileName);
               return ThreadPoolJob::jobHasFinished;
            }));

        // note: `timerCallback` will check for these jobs to be done
    }
}

I added a guard for empty string, because there are cases where the buffer can not be loaded, the file can not be written, etc. The saveBuffer function looks something like:

// TrackGUI.cpp
String TrackGUI::saveBuffer (String directoryName)
{
    jassert (track->getLength() > 0);
    AudioSampleBuffer* bufferptr = track->getAudioSampleBuffer();
    if (bufferptr != nullptr)
    {
        AudioSampleBuffer temporarySampleBufferCopy (*bufferptr);
        temporarySampleBufferCopy.applyGain (gain);
        WavAudioFormat waf;
        File file (directoryName + File::getSeparatorChar() + lblTrackName->getText() + ".wav");
        std::unique_ptr<AudioFormatWriter> afw (waf.createWriterFor (new FileOutputStream (file),
                                                                     track->getSampleRate(),
                                                                     kChannels,
                                                                     16,
                                                                     StringPairArray(),
                                                                     0));
        afw->writeFromAudioSampleBuffer (temporarySampleBufferCopy,
                                         0,
                                         track->getLength());
        afw->flush();
        
        return file.getFullPathName();
    }
    
    return "";
}

And the timerCallback then comes back to the button’s implementation for when the files are ready:

// PluginEditor.cpp
void PluginEditor::timerCallback()
{
    if (waitingForFiles && exportThreadPool.getNumJobs() == 0)
    {
        btnExport->filesReady (exportFileNames);
        waitingForFiles = false;
    }
    ...
}

calling filesReady:

// ExportButton.cpp
void ExportButton::filesReady (StringArray& fileNames)
{
    DragAndDropContainer::performExternalDragDropOfFiles (fileNames, false, this, [] () { MouseCursor::hideWaitCursor(); });
}

I experimented with setting canMoveFiles to true for this last call, but at least one DAW (Maschine) won’t accept the drop if you set it.

Alas, this implementation will let you drop generated .wav files from your plugin to a folder somewhere, or immediately into your DAW.

6

If this is a plugin id move that threadpool out of the editor, since the editor can be destroyed at any time, causing bad mojo :slight_smile:

Consider creating it as a unique ptr somewhere else and starting it if you need it.

7

Oof yes good point - thanks for pointing that out. In this case it’s running when the user starts dragging from an actual button in the interface, but indeed in any DAW, if you drag it to another track it could kill the window and the thread with it.

Are there places in the processor that could host this ThreadPool? I don’t want to bother the regular audio processing thread. I guess my question is: what is the difference between “something is in my AudioProcessor class” and “something is on the audio processing thread”?

8

You can put it in the processor, that wont mean it runs on the processor, if you start it from the ui (how else could you) it will have the lifetime of the processor which is what you want. So in your editor you can also have a check on startup to see if its running and handle that however you want, be it displaying the progress or even killing it (but youll need to be careful with that).

9

Right so if I put the function to write the files, and the threadpool, in the processor, and then from the editor just getProcessor()->startExport(); that launches the threads, then the lifetime of that threadpool is with the processor of course and will finish even if the editor closes in the meantime. Checking on whether the files have been fully written can be the same, inside timerCallback, just referencing flags and functions from the processor.

Thanks!

10

I’d probably just write a method to get the threadpool from the processor, then leave the rest of the code elsewhere since it’s not really related to the processor - potentially allowing that method to handle creating it if it doesn’t exist, but at that point, it’s just style :slight_smile: