Combining Functionality from Audio Playback Demo and Audio Recording Demo


#1

Hello,

 

I want to implement a component with the following functionality in a separate audio callback from the main plugin/audio app one:

  • Load an existing audio file
  • Record a new file (and eventually save with a user-defined filename)
  • Play the file (with transport controls)
  • Display the waveform (with zoom, scolling)

I know there are JUCE demos which do all of this, I started by adapting them so that I could have each one running in my project. That worked fine with each one individually, but I need it all happening cleanly at once. I've been trying to mash together the code from the Recording and Playback demo projects, but couldn't figure out how to put them together properly. Now I'm trying to make a new class, AudioRecorderPlayer. 

I'm successfully recording audio (using code based on the AudioRecorder class in the Recording demo), but the thumbnail isn't displaying despite the fact I'm calling thumbnail.addBlock and my component class is listening for change events (it's set to repaint when it receives such a message).

 

On the playback side, I'm able to load a file (at least, I can create a file object and pass it into setSource), but the waveform does not display and when the audio callback tries to receive samples from the file, I get an exception in ResamplingAudioSource::getNextAudioBlock because 'buffer' is empty. 

 

Here is the current state of my classes. WaveformGUI is the UI component, AudioRecorderPlayer is the backend (it's a mess, sorry).

 

class WaveformGUI  :

public Component,

public ChangeListener,

public ChangeBroadcaster,

private Slider::Listener,

private ScrollBar::Listener,

private Timer

{

public:

    enum Mode {

        IDLE = 0,

        RECORD,

        PLAY,

        NUM_MODES

    };

    

    explicit WaveformGUI (AudioDeviceManager& sharedAudioDeviceManager)

    :

    deviceManager(sharedAudioDeviceManager),

    formatManager(),


    thumbnailCache (10),

    thumbnail (512, formatManager, thumbnailCache),


    currentFilePath("Audio Recording.wav"),

    io(thumbnail),

//    recorder(thumbnail, thread),

//    player(),


    zoomSlider (new Slider()),

    scrollBar (new ScrollBar(false)),

    displayFullThumb (false)

    {

        formatManager.registerBasicFormats();

        deviceManager.addAudioCallback(&io);

        

        thumbnail.setSource(nullptr);

        thumbnail.addChangeListener(this);


        addAndMakeVisible(scrollBar);

        scrollBar->addListener (this);

        scrollBar->setRangeLimits (visibleRange);

        scrollBar->setAutoHide (false);


        currentPositionMarker.setFill (Colours::white.withAlpha (0.85f));

        addAndMakeVisible (currentPositionMarker);

    }


    ~WaveformGUI()

    {

        deviceManager.removeAudioCallback(&io);

   

        zoomSlider->removeListener (this);

        scrollBar->removeListener (this);

        thumbnail.removeChangeListener (this);

        

        io.setSource(nullptr, 0, nullptr, 0, 0);

        thumbnail.setSource (nullptr);


        scrollBar = nullptr;

        zoomSlider = nullptr;

        currentAudioFileSource = nullptr;

    }


    AudioThumbnail& getAudioThumbnail()     { return thumbnail; }


    void import()

    {

        FileChooser fc ("Choose a file to open...",

                        File::getCurrentWorkingDirectory(),

                        "*.wav",

                        true);

        fc.browseForMultipleFilesToOpen();

        const File& file = fc.getResult();

        if (! file.isDirectory())

        {

            currentFilePath = file.getFullPathName();

            io.loadFile (file, formatManager);

            updateRange();

            zoomSlider->setValue (0, dontSendNotification);

        }

    }

    

    void follow(const bool shouldFollow)

    {

        io.setFollowsTransport(shouldFollow);

    }


    void startRecording()

    {

        File newFile (File::getSpecialLocation (File::userDocumentsDirectory)

                      .getNonexistentChildFile ("Audio Recording", ".wav"));

        currentFilePath = newFile.getFullPathName();

        io.startRecording (newFile);

        setDisplayFullThumbnail (false);

    }


    void stopRecording()

    {

        io.stopRecording();

        io.loadFile(File::getSpecialLocation (File::userDocumentsDirectory)

                    .getChildFile (currentFilePath), formatManager);

        updateRange();

        setDisplayFullThumbnail (true);

    }


    void record()

    {

        if(io.isRecording())

        {

            stopRecording();

//            setMode(IDLE);

        }

        else{

            if (io.isPlaying())

            {

                io.stopPlaying();

            }

//            setMode(RECORD);

            startRecording();

        }

    }



    void play()

    {

        if (io.isPlaying())

        {

            io.stopPlaying();

//            setMode(IDLE);

        }

        else

        {

//            setMode(PLAY);

            io.setPosition (0);

            io.startPlaying();

        }


    }


    void stop()

    {

        if(io.isRecording())

        {

            stopRecording();

        }

        if (io.isPlaying())

        {

            io.stopPlaying();

        }

//        setMode(IDLE);

    }




    void setDisplayFullThumbnail (bool displayFull)

    {

        displayFullThumb = displayFull;

        repaint();

    }

    //


    File getCurrentFilePath() const noexcept                    { return currentFilePath; }


    void setZoomFactor (double amount)

    {

        if (thumbnail.getTotalLength() > 0)

        {

            const double newScale = jmax (0.001, thumbnail.getTotalLength() * (1.0 - jlimit (0.0, 0.99, amount)));

            const double timeAtCentre = xToTime (getWidth() / 2.0f);

            setRange (Range<double> (timeAtCentre - newScale * 0.5, timeAtCentre + newScale * 0.5));

        }

    }


    void setRange (Range<double> newRange)

    {

        visibleRange = newRange;

        scrollBar->setCurrentRange (visibleRange);

        updateCursorPosition();

        repaint();

    }


    void paint (Graphics& g) override

    {

        g.fillAll(juce::Colour::fromRGBA(179, 179, 179, 255));

        g.setColour(juce::Colour::fromRGBA(51, 51, 51, 255));


        if (thumbnail.getTotalLength() > 0.0)

        {

            

            Rectangle<int> thumbArea (getLocalBounds());

            thumbArea.removeFromBottom (scrollBar->getHeight() + 4);

            thumbnail.drawChannels (g, thumbArea.reduced (2),

                                    visibleRange.getStart(), visibleRange.getEnd(), 1.0f);

            

        }

        else

        {

            g.setFont (14.0f);

            g.drawFittedText ("(No audio data)", getLocalBounds(), Justification::centred, 2);

        }

    }


    void resized() override

    {

        scrollBar->setBounds (getLocalBounds().removeFromBottom (14).reduced (2));

    }


    void changeListenerCallback (ChangeBroadcaster* source) override

    {

        if (source == &thumbnail)

        {

            repaint();

        }

    }


    //

    void sliderValueChanged (Slider* slider)

    {

        if (slider == zoomSlider)

        {

            setZoomFactor (zoomSlider->getValue());

        }

    }


    void mouseDown (const MouseEvent& e) override

    {

        mouseDrag (e);

    }


    void mouseDrag (const MouseEvent& e) override

    {

        if (io.canMoveTransport())

        {

            io.setPosition (jmax (0.0, xToTime ((float) e.x)));

        }

    }


    void mouseUp (const MouseEvent&) override

    {

        if(io.isPlaying())

        {

            io.stopPlaying();

        }

        else

        {

            io.startPlaying();

        }

    }


    void mouseWheelMove (const MouseEvent&, const MouseWheelDetails& wheel) override

    {

        if (thumbnail.getTotalLength() > 0.0)

        {

            double newStart = visibleRange.getStart() - wheel.deltaX * (visibleRange.getLength()) / 10.0;

            newStart = jlimit (0.0, jmax (0.0, thumbnail.getTotalLength() - (visibleRange.getLength())), newStart);


            if (io.canMoveTransport())

                setRange (Range<double> (newStart, newStart + visibleRange.getLength()));


            if (wheel.deltaY != 0.0f)

                zoomSlider->setValue (zoomSlider->getValue() - wheel.deltaY);


            repaint();

        }

    }




private:

//    TimeSliceThread thread;

    AudioDeviceManager& deviceManager;

    AudioFormatManager formatManager;

    

    AudioThumbnailCache thumbnailCache;

    AudioThumbnail thumbnail;


    String currentFilePath;

    ScopedPointer<AudioFormatReaderSource> currentAudioFileSource;

    

    

    AudioRecorderPlayer io;

//    AudioRecorder recorder;

//    AudioTransportSource player;

    

    double sampleRate;


    ScopedPointer<Slider> zoomSlider;

    ScopedPointer<ScrollBar> scrollBar;



    Range<double> visibleRange;


    DrawableRectangle currentPositionMarker;

    bool displayFullThumb;

    

    Mode mode;


    void updateRange ()

    {

        const Range<double> newRange (0.0, thumbnail.getTotalLength());

        scrollBar->setRangeLimits (newRange);

        setRange (newRange);

        

//        startTimerHz (40);

    }


    

    float timeToX (const double time) const

    {

        return getWidth() * (float) ((time - visibleRange.getStart()) / (visibleRange.getLength()));

    }


    double xToTime (const float x) const

    {

        return (x / getWidth()) * (visibleRange.getLength()) + visibleRange.getStart();

    }


    void scrollBarMoved (ScrollBar* scrollBarThatHasMoved, double newRangeStart) override

    {

        if (scrollBarThatHasMoved == scrollBar)

            if (io.canMoveTransport())

                setRange (visibleRange.movedToStartAt (newRangeStart));

    }


    void timerCallback() override

    {

        if (io.canMoveTransport())

        {

            updateCursorPosition();

        }

        else

        {

            setRange (visibleRange.movedToStartAt (io.getCurrentPosition() - (visibleRange.getLength() / 2.0)));

        }

    }


    void updateCursorPosition()

    {

        currentPositionMarker.setVisible (io.isPlaying() || isMouseButtonDown());


        currentPositionMarker.setRectangle (Rectangle<float> (timeToX (io.getCurrentPosition()) - 0.75f, 0,

                                                              1.5f, (float) (getHeight() - scrollBar->getHeight())));

    }


    JUCE_DECLARE_NON_COPYABLE_WITH_LEAK_DETECTOR (WaveformGUI)

};

 

class AudioRecorderPlayer  : public AudioIODeviceCallback, public AudioTransportSource

{

public:

    

    AudioRecorderPlayer (AudioThumbnail& thumbnailToUpdate)

    :

    sampleRate (0),

    sourceSampleRate (0.0),

    thumbnail (thumbnailToUpdate),

    backgroundThread ("Audio Recorder Player Thread"),

    nextSampleNum (0), activeWriter (nullptr),

    bufferSize (0),


    lastGain (1.0f),

    gain (1.0f),

    playing (false),

    stopped (true),

    source (nullptr),

    resamplerSource (nullptr),

    bufferingSource (nullptr),

    positionableSource (nullptr),

    masterSource (nullptr),



    blockSize (128),

    readAheadBufferSize (0),

    isPrepared (false),

    inputStreamEOF (false),

    isFollowingTransport(false)

    {

        backgroundThread.startThread(3);

    }

    

    ~AudioRecorderPlayer()

    {

        //Transport

        if(isRecording())

        {

            stopRecording();

        }

        if(isPlaying())

        {

            stopPlaying();

        }


        releaseMasterResources();

        

        

        backgroundThread.stopThread(3);

    }

    

    TimeSliceThread * getBackgroundThread(){

        return &backgroundThread;

    }

    

    // Player

    void setSource (PositionableAudioSource* const newSource,

                    int readAheadSize, TimeSliceThread* readAheadThread,

                    double sourceSampleRateToCorrectFor, int maxNumChannels){

        if (source != newSource)

        {

            AudioSource* const oldSource = source;

            

            if (newSource != nullptr && bufferSize > 0 && sampleRate > 0){

                newSource->prepareToPlay (bufferSize, sampleRate);

            }

            

            {

                const ScopedLock sl (callbackLock);

                source = newSource;

            }

            

            if (oldSource != nullptr){

                oldSource->releaseResources();

            }

        }

        if (source == newSource)

        {

            if (source == nullptr)

                return;

            

            setSource (nullptr, 0, nullptr, 0, 0); // deselect and reselect to avoid releasing resources wrongly

        }

        

        readAheadBufferSize = readAheadSize;

        sourceSampleRate = sourceSampleRateToCorrectFor;

        

        ResamplingAudioSource* newResamplerSource = nullptr;

        BufferingAudioSource* newBufferingSource = nullptr;

        PositionableAudioSource* newPositionableSource = nullptr;

        AudioSource* newMasterSource = nullptr;

        

        ScopedPointer<ResamplingAudioSource> oldResamplerSource (resamplerSource);

        ScopedPointer<BufferingAudioSource> oldBufferingSource (bufferingSource);

        AudioSource* oldMasterSource = masterSource;

        

        if (newSource != nullptr)

        {

            newPositionableSource = newSource;

            

            if (readAheadSize > 0)

            {

                // If you want to use a read-ahead buffer, you must also provide a TimeSliceThread

                // for it to use!

                jassert (readAheadThread != nullptr);

                

                newPositionableSource = newBufferingSource

                = new BufferingAudioSource (newPositionableSource, *readAheadThread,

                                            false, readAheadSize, maxNumChannels);

            }

            

            newPositionableSource->setNextReadPosition (0);

            

            if (sourceSampleRateToCorrectFor > 0)

                newMasterSource = newResamplerSource

                = new ResamplingAudioSource (newPositionableSource, false, maxNumChannels);

            else

                newMasterSource = newPositionableSource;

            

            if (isPrepared)

            {

                if (newResamplerSource != nullptr && sourceSampleRate > 0 && sampleRate > 0)

                    newResamplerSource->setResamplingRatio (sourceSampleRate / sampleRate);

                

                newMasterSource->prepareToPlay (blockSize, sampleRate);

            }

        }

        

        {

            const ScopedLock sl (callbackLock);

            

            source = newSource;

            resamplerSource = newResamplerSource;

            bufferingSource = newBufferingSource;

            masterSource = newMasterSource;

            positionableSource = newPositionableSource;

            

            inputStreamEOF = false;

            playing = false;

        }

        

        if (oldMasterSource != nullptr)

        {

            oldMasterSource->releaseResources();

        }

    }

    

    void startPlaying()

    {

        if ((! playing) && masterSource != nullptr)

        {

            {

                const ScopedLock sl (callbackLock);

                playing = true;

                stopped = false;

                inputStreamEOF = false;

            }

            

            sendChangeMessage();

        }

    }

    

    void stopPlaying()

    {

        if (playing)

        {

            {

                const ScopedLock sl (callbackLock);

                playing = false;

            }

            

            int n = 500;

            while (--n >= 0 && ! stopped)

                Thread::sleep (2);

            

            sendChangeMessage();

        }

    }

    

    void setFollowsTransport (bool shouldFollow)

    {

        isFollowingTransport = shouldFollow;

    }

    

    bool getFollowsTransport() const

    {

        return isFollowingTransport;

    }

    

    bool canMoveTransport() const noexcept

    {

        return ! (isFollowingTransport && isPlaying());

    }

    

    void setPosition (double newPosition)

    {

        if (sampleRate > 0.0)

            setNextReadPosition ((int64) (newPosition * sampleRate));

    }

    

    double getCurrentPosition() const

    {

        if (sampleRate > 0.0)

            return getNextReadPosition() / sampleRate;

        

        return 0.0;

    }

    

    double getLengthInSeconds() const

    {

        if (sampleRate > 0.0)

            return getTotalLength() / sampleRate;

        

        return 0.0;

    }

    

    

    void setNextReadPosition (int64 newPosition)

    {

        if (positionableSource != nullptr)

        {

            if (sampleRate > 0 && sourceSampleRate > 0)

                newPosition = (int64) (newPosition * sourceSampleRate / sampleRate);

            

            positionableSource->setNextReadPosition (newPosition);

            

            if (resamplerSource != nullptr)

                resamplerSource->flushBuffers();

            

            inputStreamEOF = false;

        }

    }

    

    int64 getNextReadPosition() const

    {

        if (positionableSource != nullptr)

        {

            const double ratio = (sampleRate > 0 && sourceSampleRate > 0) ? sampleRate / sourceSampleRate : 1.0;

            return (int64) (positionableSource->getNextReadPosition() * ratio);

        }

        

        return 0;

    }

    

    int64 getTotalLength() const

    {

        const ScopedLock sl (callbackLock);

        

        if (positionableSource != nullptr)

        {

            const double ratio = (sampleRate > 0 && sourceSampleRate > 0) ? sampleRate / sourceSampleRate : 1.0;

            return (int64) (positionableSource->getTotalLength() * ratio);

        }

        

        return 0;

    }

    

    bool isLooping() const

    {

        const ScopedLock sl (callbackLock);

        return positionableSource != nullptr && positionableSource->isLooping();

    }

    

    /** Returns the source that's playing.

     May return nullptr if there's no source.

     */

    AudioSource* getCurrentSource() const noexcept      { return source; }

    

    void setGain (float newGain) noexcept

    {

        gain = newGain;

    }

    

    float getGain() const noexcept                      { return gain; }

    

    

    void prepareToPlay (int samplesPerBlockExpected, double newSampleRate)

    {

        const ScopedLock sl (callbackLock);

        

        sampleRate = newSampleRate;

        blockSize = samplesPerBlockExpected;

        

        if (masterSource != nullptr)

            masterSource->prepareToPlay (samplesPerBlockExpected, sampleRate);

        

        if (resamplerSource != nullptr && sourceSampleRate > 0)

            resamplerSource->setResamplingRatio (sourceSampleRate / sampleRate);

        

        inputStreamEOF = false;

        isPrepared = true;

    }

    

    void releaseResources()

    {

        releaseMasterResources();

    }

    

    void getNextAudioBlock (const AudioSourceChannelInfo& info)

    {

        const ScopedLock sl (callbackLock);

        

        if (masterSource != nullptr && ! stopped)

        {

            masterSource->getNextAudioBlock (info);

            

            if (! playing)

            {

                // just stopped playing, so fade out the last block..

                for (int i = info.buffer->getNumChannels(); --i >= 0;)

                    info.buffer->applyGainRamp (i, info.startSample, jmin (256, info.numSamples), 1.0f, 0.0f);

                

                if (info.numSamples > 256)

                    info.buffer->clear (info.startSample + 256, info.numSamples - 256);

            }

            

            if (positionableSource->getNextReadPosition() > positionableSource->getTotalLength() + 1

                && ! positionableSource->isLooping())

            {

                playing = false;

                inputStreamEOF = true;

                sendChangeMessage();

            }

            

            stopped = ! playing;

            

            for (int i = info.buffer->getNumChannels(); --i >= 0;)

                info.buffer->applyGainRamp (i, info.startSample, info.numSamples, lastGain, gain);

        }

        else

        {

            info.clearActiveBufferRegion();

            stopped = true;

        }

        

        lastGain = gain;

    }

    

    

    

    void audioDeviceIOCallback (const float** inputChannelData,

                                int totalNumInputChannels,

                                float** outputChannelData,

                                int totalNumOutputChannels,

                                int numSamples) override

    {

        // these should have been prepared by audioDeviceAboutToStart()...

        jassert (sampleRate > 0 && bufferSize > 0);

        


        // Player

        if(isPlaying())

        {

            const ScopedLock sl (callbackLock);

            

            if (source != nullptr)

            {

                int numActiveChans = 0, numInputs = 0, numOutputs = 0;

                

                // messy stuff needed to compact the channels down into an array

                // of non-zero pointers..

                for (int i = 0; i < totalNumInputChannels; ++i)

                {

                    if (inputChannelData[i] != nullptr)

                    {

                        inputChans [numInputs++] = inputChannelData[i];

                        if (numInputs >= numElementsInArray(inputChans)){

                            break;

                        }

                    }

                }

                

                for (int i = 0; i < totalNumOutputChannels; ++i)

                {

                    if (outputChannelData[i] != nullptr)

                    {

                        outputChans [numOutputs++] = outputChannelData[i];

                        if (numOutputs >= numElementsInArray (outputChans)){

                            break;

                        }

                    }

                }

                

                if (numInputs > numOutputs)

                {

                    // if there aren't enough output channels for the number of

                    // inputs, we need to create some temporary extra ones (can't

                    // use the input data in case it gets written to)

                    tempBuffer.setSize (numInputs - numOutputs, numSamples, false, false, true);

                    

                    for (int i = 0; i < numOutputs; ++i)

                    {

                        channels[numActiveChans] = outputChans[i];

                        memcpy (channels[numActiveChans], inputChans[i], sizeof (float) * (size_t) numSamples);

                        ++numActiveChans;

                    }

                    

                    for (int i = numOutputs; i < numInputs; ++i)

                    {

                        channels[numActiveChans] = tempBuffer.getWritePointer (i - numOutputs);

                        memcpy (channels[numActiveChans], inputChans[i], sizeof (float) * (size_t) numSamples);

                        ++numActiveChans;

                    }

                }

                else

                {

                    for (int i = 0; i < numInputs; ++i)

                    {

                        channels[numActiveChans] = outputChans[i];

                        memcpy (channels[numActiveChans], inputChans[i], sizeof (float) * (size_t) numSamples);

                        ++numActiveChans;

                    }

                    

                    for (int i = numInputs; i < numOutputs; ++i)

                    {

                        channels[numActiveChans] = outputChans[i];

                        zeromem (channels[numActiveChans], sizeof (float) * (size_t) numSamples);

                        ++numActiveChans;

                    }

                }

                

                AudioSampleBuffer buffer (channels, numActiveChans, numSamples);

                

                AudioSourceChannelInfo info (&buffer, 0, numSamples);

                getNextAudioBlock (info);

                

                

                //apply gain

                for (int i = info.buffer->getNumChannels(); --i >= 0;){

                    buffer.applyGainRamp (i, info.startSample, info.numSamples, lastGain, gain);

                    lastGain = gain;

                }

            }

            else

            {

                // We need to clear the output buffers, in case they're full of junk..

                for (int i = 0; i < totalNumOutputChannels; ++i)

                {

                    if (outputChannelData[i] != nullptr)

                    {

                        FloatVectorOperations::clear (outputChannelData[i], numSamples);

                    }

                }

            }

        }

        // Recorder

        else

        {

            if(isRecording())

            {

                const ScopedLock sl (callbackLock);

                

                if (activeWriter != nullptr)

                {

                    activeWriter->write (inputChannelData, numSamples);

                    

                    // Create an AudioSampleBuffer to wrap our incomming data, note that this does no allocations or copies, it simply references our input data

                    const AudioSampleBuffer buffer (const_cast<float**> (inputChannelData), thumbnail.getNumChannels(), numSamples);

                    thumbnail.addBlock (nextSampleNum, buffer, 0, numSamples);

                    nextSampleNum += numSamples;

                }

            }

            

            // We need to clear the output buffers, in case they're full of junk..

            for (int i = 0; i < totalNumOutputChannels; ++i)

            {

                if (outputChannelData[i] != nullptr)

                {

                    FloatVectorOperations::clear (outputChannelData[i], numSamples);

                }

            }

        }

    }

    

    void audioDeviceAboutToStart (AudioIODevice* device) override

    {

        sampleRate = device->getCurrentSampleRate();

        prepareToPlay (sampleRate, device->getCurrentBufferSizeSamples());

    }

    

    /** Implementation of the AudioIODeviceCallback method. */

    void audioDeviceStopped() override

    {

        if (source != nullptr){

            source->releaseResources();

        }

        sampleRate = 0.0;

        bufferSize = 0;

        

        tempBuffer.setSize (2, 8);

    }

    

    /** An alternative method for initialising the source without an AudioIODevice. */

    void prepareToPlay(double newSampleRate, int newBufferSize)

    {

        sampleRate = newSampleRate;

        bufferSize = newBufferSize;

        zeromem (channels, sizeof (channels));

        

        if (source != nullptr){

            source->prepareToPlay (bufferSize, sampleRate);

        }

    }

    

    //Recorder

    void startRecording (const File& file)

    {

        if (sampleRate > 0)

        {

            // Create an OutputStream to write to our destination file...

            file.deleteFile();

            ScopedPointer<FileOutputStream> fileStream (file.createOutputStream());

            if (fileStream != nullptr)

            {

                // Now create a WAV writer object that writes to our output stream...

                WavAudioFormat wavFormat;

                AudioFormatWriter* writer = wavFormat.createWriterFor (fileStream, sampleRate, 1, 16, StringPairArray(), 0);

                

                if (writer != nullptr)

                {

                    fileStream.release(); // (passes responsibility for deleting the stream to the writer object that is now using it)

                    

                    // Now we'll create one of these helper objects which will act as a FIFO buffer, and will

                    // write the data to disk on our background thread.

                    threadedWriter = new AudioFormatWriter::ThreadedWriter (writer, backgroundThread, 32768);

                    

                    // Reset our recording thumbnail

                    thumbnail.reset (writer->getNumChannels(), writer->getSampleRate());

                    nextSampleNum = 0;

                    

                    // And now, swap over our active writer pointer so that the audio callback will start using it..

                    const ScopedLock sl (callbackLock);

                    activeWriter = threadedWriter;

                }

            }

        }

    }

    

    void stopRecording()

    {

        // First, clear this pointer to stop the audio callback from using our writer object..

        {

            const ScopedLock sl (callbackLock);

            activeWriter = nullptr;

        }

        

        // Now we can delete the writer object. It's done in this order because the deletion could

        // take a little time while remaining data gets flushed to disk, so it's best to avoid blocking

        // the audio callback while this happens.

        threadedWriter = nullptr;


    }

    

    bool isRecording() const

    {

        return activeWriter != nullptr;

    }

    

    bool isPlaying() const

    {

        return playing;

    }

    

    void loadFile(const File& file, AudioFormatManager& formatManager)

    {

        // unload the previous file source and delete it..

        stopRecording();

        stopPlaying();

        setSource (nullptr, 0, nullptr, 0, 0);

        

        AudioFormatReader* reader = formatManager.createReaderFor (file);

        

        if (reader != nullptr)

        {

            // ..and plug it into our transport source

            setSource (new AudioFormatReaderSource (reader, false),

                          32768,                   // tells it to buffer this many samples ahead

                          &backgroundThread,                 // this is the background thread to use for reading-ahead

                          reader->sampleRate, reader->numChannels);     // allows for sample rate correction

            thumbnail.setSource (new FileInputSource (file));

        }

    }

    

private:

    //==============================================================================

    // Shared

    double sampleRate, sourceSampleRate;

    CriticalSection callbackLock;

    AudioThumbnail& thumbnail;

    

    // Recorder

    TimeSliceThread backgroundThread; // the thread that will write our audio data to disk

    ScopedPointer<AudioFormatWriter::ThreadedWriter> threadedWriter; // the FIFO used to buffer the incoming data

    int64 nextSampleNum;

    

//    CriticalSection writeLock;

    AudioFormatWriter::ThreadedWriter* volatile activeWriter;

    

    // Player

//    CriticalSection readLock;

    int bufferSize;

    float* channels [128];

    float* outputChans [128];

    const float* inputChans [128];

    AudioSampleBuffer tempBuffer;

    float volatile lastGain, gain;

    bool volatile playing, stopped;

    

    PositionableAudioSource* source;

    ResamplingAudioSource* resamplerSource;

    BufferingAudioSource* bufferingSource;

    PositionableAudioSource* positionableSource;

    AudioSource* masterSource;

    

    int blockSize, readAheadBufferSize;

    bool volatile isPrepared, inputStreamEOF;

    bool isFollowingTransport;

    

    void releaseMasterResources()

    {

        const ScopedLock sl (callbackLock);

        

        if (masterSource != nullptr)

            masterSource->releaseResources();

        

        isPrepared = false;

    }

    

    JUCE_DECLARE_NON_COPYABLE_WITH_LEAK_DETECTOR (AudioRecorderPlayer)

};

I think at least part of the problem is that I'm not setting the source for the transport properly. I've been stuck on this for a few days now so thanks in advance to anyone who can help me. Even just knowing the correct classes to inhereit from would be a good start. I figured I need 
AudioIODeviceCallback for the recording part, and AudioTransportSource to play and load files. 


#2

On the playback side, I'm able to load a file (at least, I can create a file object and pass it into setSource), but the waveform does not display and when the audio callback tries to receive samples from the file, I get an exception in ResamplingAudioSource::getNextAudioBlock because 'buffer' is empty. 

The waveform displayed fine here when loading a WAV file. 

If you post the project with the rest of the UI controls on as a zip somewhere I'll have a quick look...


#3

On the playback side, I'm able to load a file (at least, I can create a file object and pass it into setSource), but the waveform does not display and when the audio callback tries to receive samples from the file, I get an exception in ResamplingAudioSource::getNextAudioBlock because 'buffer' is empty. 

The waveform displayed fine here when loading a WAV file. 

If you post the project with the rest of the UI controls on as a zip somewhere I'll have a quick look...


#4

Thanks so much for your help on this, bazrush! If anyone else is trying to accomplish the same thing, feel free to message me for some updated code.


#5

Hello again,

 

Bazrush got me on the right track wrt my AudioRecorderPlayer class -- it works (though still I can't claim to fully understand all the code I borrowed from the demos). 

 

AudioRecorderPlayer.h:

#ifndef AUDIORECORDERPLAYER_H_INCLUDED

#define AUDIORECORDERPLAYER_H_INCLUDED


#include "JuceHeader.h"


class AudioRecorderPlayer  : public AudioSourcePlayer, public ActionBroadcaster,

                             public ChangeBroadcaster, public ChangeListener

{

public:

    

    AudioRecorderPlayer (AudioFormatManager& audioFormatManager, AudioThumbnail& thumbnailToUpdate);

    

    ~AudioRecorderPlayer();

    

    TimeSliceThread* getBackgroundThread();


    void setSource(AudioSource* newSource);

    

    void startPlaying();

    

    void stopPlaying();

    

    void setFollowsTransport (bool shouldFollow);

    

    bool getFollowsTransport() const;

    

    bool canMoveTransport() const noexcept;

    

    void setPosition (double newPosition);// in seconds

    

    double getCurrentPosition() const;// in seconds


    double getSamplingRate();

    

    double getFileSamplingRate();

    

    bool getFileSaveState();

    

    const String& getCurrentFilePath() const noexcept;

    

    double getLengthInSeconds() const;

    

    void setNextReadPosition (int64 newPosition);// in samples

    

    int64 getNextReadPosition() const;// in samples

    

    int64 getLengthInSamples() const;//in samples

    

    bool isLooping() const;

    

    /** Returns the source that's playing.

     May return nullptr if there's no source.

     */

    AudioSource* getCurrentSource() const noexcept;

    

    void setGain (float newGain) noexcept;

    

    float getGain() const noexcept;

    

    void prepareToPlay (int newBufferSize, double newSampleRate);

    

    

    void audioDeviceIOCallback (const float** inputChannelData,

                                int totalNumInputChannels,

                                float** outputChannelData,

                                int totalNumOutputChannels,

                                int numSamples) override;

    

    void audioDeviceAboutToStart (AudioIODevice* device) override;


    void audioDeviceStopped() override;

    

    void startRecording();

    

    void stopRecording();

    

    bool isRecording() const;

    

    bool isPlaying() const;

    

    bool loadFile(const bool loadNewlyRecordedFile);

    

    bool saveFile();

    

    void reset();

    

private:

    //==============================================================================

    double sampleRate, fileSampleRate;

    CriticalSection callbackLock;

    AudioThumbnail& thumbnail;

    

    TimeSliceThread backgroundThread; // the thread that will write our audio data to disk

    ScopedPointer<AudioFormatWriter::ThreadedWriter> threadedWriter; // the FIFO used to buffer the incoming data

    int64 nextSampleNum;

    

    const String tempFileName;

    String currentFilePath;

    bool currentFileHasBeenSaved;

    AudioFormatManager& formatManager;

    AudioFormatWriter::ThreadedWriter* volatile activeWriter;

    

    int bufferSize;

    float* channels [128];

    float* outputChans [128];

    const float* inputChans [128];

    AudioSampleBuffer tempBuffer;

    float volatile lastGain, gain;

    

    AudioSource* audioSource;

    AudioTransportSource transport;

    ScopedPointer<AudioFormatReaderSource> currentAudioFileSource;

    bool isFollowingTransport;

    

    void changeListenerCallback(ChangeBroadcaster* broadcastSource) override;

    

    JUCE_DECLARE_NON_COPYABLE_WITH_LEAK_DETECTOR (AudioRecorderPlayer)

};



#endif  // AUDIORECORDERPLAYER_H_INCLUDED

 

AudioRecorderPlayer.cpp

#include "audioRecorderPlayer.h"


//=================================================================================

// AudioRecorderPlayer ============================================================

//=================================================================================


// Public =========================================================================


AudioRecorderPlayer::AudioRecorderPlayer(AudioFormatManager& audioFormatManager, AudioThumbnail& thumbnailToUpdate)

:

sampleRate(0),

fileSampleRate(0.0),

thumbnail(thumbnailToUpdate),

backgroundThread("AudioRecorderPlayer Thread"),

nextSampleNum(0),

tempFileName("Audio_Clip.wav"),

currentFilePath(""),

currentFileHasBeenSaved(false),

formatManager(audioFormatManager),

activeWriter(nullptr),

bufferSize(0),


lastGain(1.0f),

gain(1.0f),

audioSource(nullptr),

currentAudioFileSource(nullptr)

{

    File tempFile (File::getSpecialLocation (File::tempDirectory)

                   .createTempFile(tempFileName));

    transport.addChangeListener(this);

    reset();

    audioSource= &transport;

    backgroundThread.startThread(3);

}


AudioRecorderPlayer::~AudioRecorderPlayer()

{

    if(isRecording())

    {

        stopRecording();

    }

    if(isPlaying())

    {

        stopPlaying();

    }

    reset();

    backgroundThread.stopThread(3);

    transport.removeChangeListener(this);

    

    threadedWriter = nullptr;

    currentAudioFileSource = nullptr;

    

    File tempFile (File::getSpecialLocation (File::tempDirectory)

                   .getChildFile(tempFileName));

    tempFile.deleteFile();

}


TimeSliceThread* AudioRecorderPlayer::getBackgroundThread()

{

    return &backgroundThread;

}


void AudioRecorderPlayer::setSource(AudioSource* newSource)

{

    if (audioSource!= newSource)

    {

        AudioSource* const oldSource = audioSource;

        

        if (newSource != nullptr && bufferSize > 0 && sampleRate > 0)

            newSource->prepareToPlay (bufferSize, sampleRate);

        

        {

            const ScopedLock sl (callbackLock);

            audioSource= newSource;

        }

        

        if (oldSource != nullptr)

            oldSource->releaseResources();

    }

}



void AudioRecorderPlayer::startPlaying()

{

    transport.start();

}


void AudioRecorderPlayer::stopPlaying()

{

    transport.stop();

}


void AudioRecorderPlayer::setFollowsTransport (bool shouldFollow)

{

    isFollowingTransport = shouldFollow;

}


bool AudioRecorderPlayer::getFollowsTransport() const

{

    return isFollowingTransport;

}


bool AudioRecorderPlayer::canMoveTransport() const noexcept

{

    return ! (isFollowingTransport && isPlaying());

}


void AudioRecorderPlayer::setPosition (double newPosition)// in seconds

{

    transport.setPosition(newPosition);

}


double AudioRecorderPlayer::getCurrentPosition() const// in seconds

{

    return transport.getCurrentPosition();

}


double AudioRecorderPlayer::getLengthInSeconds() const

{

    return transport.getLengthInSeconds();

}


double AudioRecorderPlayer::getSamplingRate()

{

    return sampleRate;

}


double AudioRecorderPlayer::getFileSamplingRate()

{

    return fileSampleRate;

}


bool AudioRecorderPlayer::getFileSaveState()

{

    return currentFileHasBeenSaved;

}


const String& AudioRecorderPlayer::getCurrentFilePath() const noexcept

{

    return currentFilePath;

}


void AudioRecorderPlayer::setNextReadPosition (int64 newPosition)// in samples

{

    transport.setPosition(newPosition);

}


int64 AudioRecorderPlayer::getNextReadPosition() const// in samples

{

    return transport.getNextReadPosition();

}


int64 AudioRecorderPlayer::getLengthInSamples() const//in samples

{

    return transport.getTotalLength();

}


bool AudioRecorderPlayer::isLooping() const

{

    return transport.isLooping();

}


/** Returns the source that's playing.

 May return nullptr if there's no source.

 */

AudioSource* AudioRecorderPlayer::getCurrentSource() const noexcept

{

    return currentAudioFileSource;

}


void AudioRecorderPlayer::setGain (float newGain) noexcept

{

    gain = newGain;

}


float AudioRecorderPlayer::getGain() const noexcept

{

    return gain;

}


void AudioRecorderPlayer::prepareToPlay (int newBufferSize, double newSampleRate)

{

    sampleRate = newSampleRate;

    bufferSize = newBufferSize;

    zeromem (channels, sizeof (channels));

    

    if(audioSource != nullptr)

    {

        audioSource->prepareToPlay(bufferSize, sampleRate);

    }

}



void AudioRecorderPlayer::audioDeviceIOCallback (const float** inputChannelData,

                            int totalNumInputChannels,

                            float** outputChannelData,

                            int totalNumOutputChannels,

                            int numSamples)

{

    // these should have been prepared by audioDeviceAboutToStart()...

    jassert (sampleRate > 0 && bufferSize > 0);

    

    

    // Player

    if(isPlaying())

    {

        const ScopedLock sl (callbackLock);

        

        if (audioSource!= nullptr)

        {

            int numActiveChans = 0, numInputs = 0, numOutputs = 0;

            

            // messy stuff needed to compact the channels down into an array

            // of non-zero pointers..

            for (int i = 0; i < totalNumInputChannels; ++i)

            {

                if (inputChannelData[i] != nullptr)

                {

                    inputChans [numInputs++] = inputChannelData[i];

                    if (numInputs >= numElementsInArray(inputChans)){

                        break;

                    }

                }

            }

            

            for (int i = 0; i < totalNumOutputChannels; ++i)

            {

                if (outputChannelData[i] != nullptr)

                {

                    outputChans [numOutputs++] = outputChannelData[i];

                    if (numOutputs >= numElementsInArray (outputChans)){

                        break;

                    }

                }

            }

            

            if (numInputs > numOutputs)

            {

                // if there aren't enough output channels for the number of

                // inputs, we need to create some temporary extra ones (can't

                // use the input data in case it gets written to)

                tempBuffer.setSize (numInputs - numOutputs, numSamples, false, false, true);

                

                for (int i = 0; i < numOutputs; ++i)

                {

                    channels[numActiveChans] = outputChans[i];

                    memcpy (channels[numActiveChans], inputChans[i], sizeof (float) * (size_t) numSamples);

                    ++numActiveChans;

                }

                

                for (int i = numOutputs; i < numInputs; ++i)

                {

                    channels[numActiveChans] = tempBuffer.getWritePointer (i - numOutputs);

                    memcpy (channels[numActiveChans], inputChans[i], sizeof (float) * (size_t) numSamples);

                    ++numActiveChans;

                }

            }

            else

            {

                for (int i = 0; i < numInputs; ++i)

                {

                    channels[numActiveChans] = outputChans[i];

                    memcpy (channels[numActiveChans], inputChans[i], sizeof (float) * (size_t) numSamples);

                    ++numActiveChans;

                }

                

                for (int i = numInputs; i < numOutputs; ++i)

                {

                    channels[numActiveChans] = outputChans[i];

                    zeromem (channels[numActiveChans], sizeof (float) * (size_t) numSamples);

                    ++numActiveChans;

                }

            }

            

            AudioSampleBuffer buffer (channels, numActiveChans, numSamples);

            

            AudioSourceChannelInfo info (&buffer, 0, numSamples);

            audioSource->getNextAudioBlock (info);

            

            //apply gain

            for (int i = info.buffer->getNumChannels(); --i >= 0;){

                buffer.applyGainRamp (i, info.startSample, info.numSamples, lastGain, gain);

            }

            lastGain = gain;

        }

        else

        {

            stopPlaying();

        }

        return;

    }

    // Recorder

    else if(isRecording())

    {

        const ScopedLock sl (callbackLock);

        

        if (activeWriter != nullptr)

        {

            activeWriter->write (inputChannelData, numSamples);

            

            // Create an AudioSampleBuffer to wrap our incomming data, note that this does no allocations or copies, it simply references our input data

            const AudioSampleBuffer buffer (const_cast<float**> (inputChannelData), thumbnail.getNumChannels(), numSamples);

            thumbnail.addBlock (nextSampleNum, buffer, 0, numSamples);

            nextSampleNum += numSamples;

        }

    }

    // We need to clear the output buffers, in case they're full of junk..

    for (int i = 0; i < totalNumOutputChannels; ++i)

    {

        if (outputChannelData[i] != nullptr)

        {

            FloatVectorOperations::clear (outputChannelData[i], numSamples);

        }

    }

}


void AudioRecorderPlayer::audioDeviceAboutToStart (AudioIODevice* device)

{

    prepareToPlay(device->getCurrentBufferSizeSamples(), device->getCurrentSampleRate());

}


void AudioRecorderPlayer::audioDeviceStopped()

{

    if (audioSource!= nullptr){

        audioSource->releaseResources();

    }

    sampleRate = 0.0;

    bufferSize = 0;

    

    tempBuffer.setSize (2, 8);

}


void AudioRecorderPlayer::startRecording()

{

    if (sampleRate > 0)

    {

        File file = File::getSpecialLocation(File::tempDirectory).getChildFile(tempFileName);

        currentFilePath = file.getFullPathName();

        std::cout << "Recording to " + currentFilePath << std::endl;

        // Create an OutputStream to write to our destination file...

        file.deleteFile();

        ScopedPointer<FileOutputStream> fileStream (file.createOutputStream());

        if (fileStream != nullptr)

        {

            // Now create a WAV writer object that writes to our output stream...

            WavAudioFormat wavFormat;

            AudioFormatWriter* writer = wavFormat.createWriterFor (fileStream, sampleRate, 1, 16, StringPairArray(), 0);

            

            if (writer != nullptr)

            {

                fileStream.release(); // (passes responsibility for deleting the stream to the writer object that is now using it)

                

                // Now we'll create one of these helper objects which will act as a FIFO buffer, and will

                // write the data to disk on our background thread.

                threadedWriter = new AudioFormatWriter::ThreadedWriter (writer, backgroundThread, 32768);

                

                // Reset our recording thumbnail

                thumbnail.reset (writer->getNumChannels(), writer->getSampleRate());

                nextSampleNum = 0;

                

                // And now, swap over our active writer pointer so that the audio callback will start using it..

                const ScopedLock sl (callbackLock);

                activeWriter = threadedWriter;

            }

        }

    }

}


void AudioRecorderPlayer::stopRecording()

{

    currentFileHasBeenSaved = false;

    

    // First, clear this pointer to stop the audio callback from using our writer object..

    {

        const ScopedLock sl (callbackLock);

        activeWriter = nullptr;

    }

    

    // Now we can delete the writer object. It's done in this order because the deletion could

    // take a little time while remaining data gets flushed to disk, so it's best to avoid blocking

    // the audio callback while this happens.

    threadedWriter = nullptr;

    loadFile(true);

    currentFileHasBeenSaved = false;

}


bool AudioRecorderPlayer::isRecording() const

{

    return activeWriter != nullptr;

}


bool AudioRecorderPlayer::isPlaying() const

{

    return transport.isPlaying();

}


bool AudioRecorderPlayer::loadFile(const bool loadNewlyRecordedFile = false)

{

    File file;

    if(loadNewlyRecordedFile) // We just stopped recording

    {

        // TODO This should maybe be user temp folder instead..

        file = File(currentFilePath);

    }

    else

    {

        FileChooser fc("Choose a file to open...",

                        File::getSpecialLocation(File::SpecialLocationType::userDocumentsDirectory),

                        "*.wav",

                        true);

        fc.browseForMultipleFilesToOpen();

        file = fc.getResult();

        currentFilePath = file.getFullPathName();

        // unload the previous file source and delete it..


    }

    if (!file.isDirectory())

    {

        AudioFormatReader * reader = formatManager.createReaderFor (file);

        

        if (reader != nullptr)

        {

            // ..and plug it into our transport source

            const ScopedLock sl(callbackLock);

            ScopedPointer<AudioFormatReaderSource> newAudioFileSource = new AudioFormatReaderSource (reader, true);

            transport.setSource (newAudioFileSource,

                                 32768,                   // tells it to buffer this many samples ahead

                                 &backgroundThread,                 // this is the background thread to use for reading-ahead

                                 reader->sampleRate, reader->numChannels);     // allows for sample rate correction

            fileSampleRate = reader->sampleRate;

            currentAudioFileSource = newAudioFileSource;

            thumbnail.setSource (new FileInputSource (file));

            newAudioFileSource = nullptr;

            


            currentFileHasBeenSaved = false;

            sendActionMessage("Audio Available");

            return true;

        }

    }

    return false;

}


bool AudioRecorderPlayer::saveFile()

{

    if(!currentFileHasBeenSaved)

    {

        stopRecording();

        stopPlaying();

        

        

        //TODO create preference for directory for saving files

        //TODO save labeling data alongside them

        FileChooser fc ("Choose a location to save file...",

                        File::getSpecialLocation(File::SpecialLocationType::userDocumentsDirectory));

        fc.browseForFileToSave(true);

        const File& file = fc.getResult();

        if(!file.isDirectory())

        {

            bool success = false;

            

            file.deleteFile();

            ScopedPointer<FileOutputStream> fileStream(file.createOutputStream());

            if(fileStream != nullptr)

            {

                // Now create a WAV writer object that writes to our output stream...

                WavAudioFormat wavFormat;

                ScopedPointer<AudioFormatWriter> writer = wavFormat.createWriterFor(fileStream, sampleRate, 1, 16, StringPairArray(), 0);

                if (writer != nullptr)

                {

                    fileStream.release(); // (passes responsibility for deleting the stream to the writer object that is now using it)

                    currentAudioFileSource->setNextReadPosition(0);

                    success = writer->writeFromAudioSource(*currentAudioFileSource,

                                                           currentAudioFileSource->getTotalLength(),

                                                           4096);

                    writer = nullptr;

                }

            }

            

            if(success)

            {

                currentFilePath = file.getFullPathName();

                currentFileHasBeenSaved = true;

            }

            // TODO check if this is necessary

            else

            {

                currentFileHasBeenSaved = false;

            }

        }

    }

    return currentFileHasBeenSaved;

}


void AudioRecorderPlayer::reset()

{

    transport.setSource(nullptr);

    assert(transport.getTotalLength() == 0);

    sendActionMessage("No Audio Available");

}


// Private ========================================================================


void AudioRecorderPlayer::changeListenerCallback(ChangeBroadcaster * broadcastSource)

{

    if(broadcastSource == &transport)

    {

        if(transport.hasStreamFinished()){

            sendActionMessage("Stop Playing");

        }

    }

}

Sorry for all the extra line breaks, I blame Xcode.

 

Allow me to explain the overall structure of my app, and some of my motivations for certain decisions:

 

First, my project is a prototype for what will eventually run as a plugin. When testing basic functionality I find that working within the AudioApplication template is much more convenient though. Since there are some differences between the audio app and plugin templates, I tried to make my code as portable as possible by putting it all in one class derived from Component, so that it would just be a matter of adding an instance of that class to the main window regardless of whether I'm using it in a plugin or a stand alone audio app. I'll refer to this class as the 'primary class'. An instance of the primary class is owned by the MainContentComponent, and the primary class owns an instance of the AudioRecorderPlayer as well as an instance of my audio analyzer. 

 

After working on other parts of the project for some time, I've come to the point where I need to integrate the AudioRecorderPlayer class with my main AudioAppComponent callback, which is meant to run the audio analysis code. At first I thought that since I call 'audioDeviceManager.addAudioCallback(&io);', where 'io' is an instance of the AudioRecorderPlayer class, JUCE would just automagically run both callbacks sequentially. This sort of works in the sense that I can use the AudioRecorderPlayer without any issues. The problem is that when I add the AudioRecorderPlayer's callback I lose the AudioAppComponent's callback -- it only executes once and then mysteriously dies. I'm sure there's a reason for this that I just don't understand... 

 

Now, there's really no reason that I need to have two separate audio callbacks, it would just be a convenient way of switching behavior in the different modes of my program. Basically there are three main IO functions, two of which are currently handled splendidly by the AudioRecorderPlayer:

 

1) Recording and allowing playback of a new audio file (AudioRecorderPlayer, makes use of a background thread like the JUCE recording demo)

2) Loading and allowing playback of an existing audio file (AudioRecorderPlayer, makes use of transport and audio source objects like the JUCE playback demo)

3) Processing real-time input from the audio device (I intended this to happen in the main audio callback of the app/plugin)

 

These three modes are all mutually exclusive -- I want my app to be able to do all of these things from a single GUI, but it only makes sense to be doing one thing at a time. Thus I was hoping that I could just swap callbacks as needed and ignore the AudioRecorderPlayer entirely while I'm processing real-time input. Letting the AudioRecorderPlayer handle all that stuff on its own was a nice way for me to compartmentalize things. 

 

What's the best practice here? I'm beginning to think that I might need to just move everything into the MainContentComponent and use its callback for all three tasks, but then I would be pouring all the guts of this nicely self-contained AudioRecorderPlayer into my primary class. Should I try to alter AudioRecorderPlayer so that it relies on another class for its callback? I welcome all suggestions!

 

Thanks