Using quicktime optionally

[chkn]

Hi,
i use Quicktime to read MP3 files on Windows via formatManager.registerFormat(new QuickTimeAudioFormat(),false);
So my question is, what happens if my application runs on a non-quicktime PC?
And how can i detect if Quicktime is not installed, to display a alert message like “For MP3 support, please install Apple Quicktime on your PC”

[jules]

Well, it should just fail to open any quicktime files if it’s not installed. You can check using QuickTimeMovieComponent::isQuickTimeAvailable()

[chkn]

Thanks!

[chkn]

Can it be that reading MP3 via Quicktime is very slow?, it needs serveral minutes for a normal sized mp3.
Is it because that MovieAudioExtractionFillBuffer is used sample by sample, can it be used for working with blocks mhhh…

Also the quicktime methods throwing some exceptions (deep inside coreaudio.dll etc.) in the beginning, but which doesn’t impact the result.

I’m just using

[code]
bool loadFromFile(File file)
{
bool success=false;

	AudioFormatManager formatManager;
	formatManager.registerBasicFormats();
	formatManager.registerFormat(new QuickTimeAudioFormat(),false);
	AudioFormatReader* reader = formatManager.createReaderFor (file);

	
	if (reader != 0)
	{
                    formatName=reader->getFormatName();

		int readlength=0;
		if ( reader->lengthInSamples > 2147483646 ) 
		{ 
			readlength=2147483646; 
		} 
		else 
		{ 
			readlength=(int)reader->lengthInSamples; 
		};

		if (  readlength > 0)
		{
			
			setSize(reader->numChannels,readlength,false,false,false);
			set_samplerate(reader->sampleRate);
			readFromAudioReader(reader,0,readlength,0,true,true);
			success=true;
		} else
		{
			jassertfalse;
		};

	};

	deleteAndZero(reader);

	return success;
}[/code]

[jules]

Is it because that MovieAudioExtractionFillBuffer is used sample by sample, can it be used for working with blocks mhhh…

It’s certainly going to be very slow at seeking - if you were making it seek before each read, that’d absolutely hammer the performance.

[chkn]

mhh not sure what you mean, what can i do?

[jules]

You’d just have to be careful to only read sequentially, and not skip around.

[chkn]

All i do is to load the whole file into a AudioSampleBuffer in one step, there is no skipping around

myAudioSampleBuffer.readFromAudioReader(reader,0,readlength,0,true,true);

[jules]

Not sure then, QT is a bit of a black box when it comes to understanding what it’s doing…

[chkn]

I’ve got a massive speed up (seconds instead of minutes!) when i change the “samplesPerFrame” parameter (which is used by MovieAudioExtractionFillBuffer) when importing a MP3 , and it seems to run without problems!
Would be great if you have a short look at this ( Quicktime i think is a well acceptable solution for end users who want to use MP3s )

in QTAudioReader::QTAudioReader

        samplesPerFrame = (int) (GetMediaDecodeDuration (media) / GetMediaSampleCount (media));

	//test
	samplesPerFrame = (int) 1024;

[jules]

That’s definitely not a good value to hack! It’s describing a property of the file, so changing it will break the behaviour for some files. Maybe try have looking at the places that it gets used if you’re trying to work out why this is improving performance, rather than changing the value itself?

[chkn]

The performance problem is that MovieAudioExtractionFillBuffer always called for 1 sample (samplesPerFrame= 1 = (int) (GetMediaDecodeDuration (media) / GetMediaSampleCount (media));), so if the file has 200000 samples, MovieAudioExtractionFillBuffer is called 200 000 times , and thats why importing a MP3 file takes 10 Minutes, but i can be used to loading more samples.

from the apple-docs http://developer.apple.com/quicktime/audioextraction.html

Note that MovieAudioExtractionFillBuffer will extract as many of the requested PCM frames as it can, given the limits of the buffer(s) supplied, and the limits of the input movie. ioNumFrames will be updated with the exact number of valid frames being returned. When there is no more audio to extract from the movie, MovieAudioExtractionFillBuffer will continue to return noErr, but no audio data will be returned. outFlags will have the kQTMovieAudioExtractionComplete bit set in this case, as shown in Listing 7.

[jules]

Well maybe something like this would make more sense?

[code] int samplesWanted = 2048;
int framesToDo = jmax (2, samplesWanted / samplesPerFrame);

    bufferList->mBuffers[0].mDataByteSize = inputStreamDesc.mBytesPerFrame * framesToDo;
    dataBuffer.malloc (bufferList->mBuffers[0].mDataByteSize);
    bufferList->mBuffers[0].mData = dataBuffer;

    UInt32 outFlags = 0;
    UInt32 actualNumFrames = framesToDo;
    OSStatus err = MovieAudioExtractionFillBuffer (extractor, &actualNumFrames,
                                                   bufferList, &outFlags);

    lastSampleRead = sampleNum + actualNumFrames * samplesPerFrame;

[/code]

[chkn]

yes it looks right, bit it doesn’t work (or it takes ages, or a deadlock?) ,
its seems Quicktime doesn’t like it when you change bufferList-parameters (and memory-locations) afterwards?!
I think its better to allocate a big enough amount of memory in the beginning, without relocating.

I have some mystery first chance exceptions (even with original sourcecode, deep in quicktime, with a fantasy Call Stack in VS ), in the beginning of a decode-session, but it doesn’t seem to affect the the decoding process (because it works!!!), maybe Quicktime holds pointers (or whatever) to the memory…

[code]extern “C” _CRTIMP void __cdecl _free_dbg(
void * pUserData,
int nBlockUse
)
{
/* lock the heap
*/
_mlock(_HEAP_LOCK);

    __try {
        /* allocate the block
         */
        _free_dbg_nolock(pUserData, nBlockUse);
    }
    __finally {
        /* unlock the heap
         */
        _munlock(_HEAP_LOCK);                 <-------------------  First-chance exception 
    }

}[/code]

 	kernel32.dll!7593e124() 	
 	CoreAudioToolbox.dll!65dae0a0() 	
 	CoreAudioToolbox.dll!65b5ff38() 	
 	CoreAudioToolbox.dll!65c85849() 	
 	CoreAudioToolbox.dll!65c6c6de() 	
 	ntdll.dll!77303029() 	
 	ntdll.dll!77303029() 	
 	ntdll.dll!77303029() 	
 	ntdll.dll!772daaad() 	
 	CoreAudioToolbox.dll!65c760cc() 	
 	kernel32.dll!759a3f01() 	
 	CoreAudioToolbox.dll!65dd28ce() 	
 	CoreAudioToolbox.dll!65d4c5de() 	
 	CoreAudioToolbox.dll!65d12510() 	
 	CoreAudioToolbox.dll!65dd132b() 	
 	CoreAudioToolbox.dll!65d031b3() 	
 	CoreAudioToolbox.dll!65d032ba() 	
 	QuickTime.qts!6e7fb726() 	
 	QuickTime.qts!6e853111() 	
 	QuickTime.qts!6e7fb731() 	
 	QuickTime.qts!6e853210() 	
 	QuickTime.qts!6f02324f() 	
 	QuickTime.qts!6e853111() 	
 	QuickTime.qts!6e7fb726() 	
 	QuickTime.qts!6e7fb726() 	
 	QuickTime.qts!6e90c9f0() 	
 	QuickTime.qts!6e7fb726() 	
>	My_Player_Debug.exe!_free_dbg(void * pUserData=0x00000004, int nBlockUse=9484552)  Line 1263 + 0x7 bytes	C++
 	My_Player_Debug.exe!_free_dbg(void * pUserData=0x00000004, int nBlockUse=9484552)  Line 1260 + 0xc bytes	C++
 	My_Player_Debug.exe!operator delete(void * pUserData=0x031d10a0)  Line 57 + 0x7 bytes	C++
 	My_Player_Debug.exe!operator delete[](void * p=0x077c004c)  Line 21 + 0x9 bytes	C++
 	My_Player_Debug.exe!juce::StringHolder::release(juce::StringHolder * const b=0x0063beff)  Line 97 + 0xf bytes	C++
 	04d3fe48()	
 	My_Player_Debug.exe!juce::CriticalSection::exit()  Line 88	C++
 	My_Player_Debug.exe!juce::ScopedLock::~ScopedLock()  Line 78 + 0x16 bytes	C++
	ffffffff()	
 	ntdll.dll!7732d45f() 	
 	My_Player_Debug.exe!juce::MemoryOutputStream::flush()  Line 61	C++
 	My_Player_Debug!juce::Component::SafePointer<juce::ComboBox>::~SafePointer<juce::ComboBox>()  Line 1955 + 0x4d bytes	C++
 	My_Player_Debug!juce::Component::SafePointer<juce::ComboBox>::~SafePointer<juce::ComboBox>()  Line 1955 + 0x4d bytes	C++
 	My_Player_Debug!juce::Component::SafePointer<juce::ComboBox>::~SafePointer<juce::ComboBox>()  Line 1955 + 0x4d bytes	C++

[jules]

Ok… I’ve not got time to try this myself, but how about:

In the constructor, create a bigger buffer:

bufferList->mBuffers[0].mDataByteSize = jmax (4096, (UInt32) (samplesPerFrame * inputStreamDesc.mBytesPerFrame) + 16);

and then:

[code] int framesToDo = bufferList->mBuffers[0].mDataByteSize / inputStreamDesc.mBytesPerFrame;
bufferList->mBuffers[0].mDataByteSize = inputStreamDesc.mBytesPerFrame * framesToDo;

    UInt32 outFlags = 0;
    UInt32 actualNumFrames = framesToDo;
    OSStatus err = MovieAudioExtractionFillBuffer (extractor, &actualNumFrames, bufferList, &outFlags);

    lastSampleRead = sampleNum + actualNumFrames * samplesPerFrame;

[/code]

[chkn]

yes, but it also affect the readSamples() method
in the end, i integrated the readFrames into readSamples.
Also you have to check that kQTMovieAudioExtractionComplete flaq, because the file-length in the end can be different as assumed ( 10-20 samples )

And then we getting a endless loop, because Quicktime modifiers the bufferList->mBuffers[0].mDataByteSize, which is used to calculated the Frames to load.
So we have to check the flaq, in fill the rest with zero ( nut sure how to do that…)
( the old code didn’t check this, add ads always the same last sample )

class QTAudioReader     : public AudioFormatReader
{
public:
    QTAudioReader (InputStream* const input_, const int trackNum_)
        : AudioFormatReader (input_, TRANS (quickTimeFormatName)),
          ok (false),
          movie (0),
          trackNum (trackNum_),
          lastSampleRead (0),
          lastThreadId (0),
          extractor (0),
          dataHandle (0)
    {
        bufferList.calloc (256, 1);

#if JUCE_WINDOWS
        if (InitializeQTML (0) != noErr)
            return;
#endif
        if (EnterMovies() != noErr)
            return;

        bool opened = juce_OpenQuickTimeMovieFromStream (input_, movie, dataHandle);

        if (! opened)
            return;

        {
            const int numTracks = GetMovieTrackCount (movie);
            int trackCount = 0;

            for (int i = 1; i <= numTracks; ++i)
            {
                track = GetMovieIndTrack (movie, i);
                media = GetTrackMedia (track);

                OSType mediaType;
                GetMediaHandlerDescription (media, &mediaType, 0, 0);

                if (mediaType == SoundMediaType
                     && trackCount++ == trackNum_)
                {
                    ok = true;
                    break;
                }
            }
        }

        if (! ok)
            return;

        ok = false;

        lengthInSamples = GetMediaDecodeDuration (media);
        usesFloatingPointData = false;

        samplesPerFrame = (int) (GetMediaDecodeDuration (media) / GetMediaSampleCount (media));

        //test
//        samplesPerFrame = (int) 1024;

        trackUnitsPerFrame = GetMovieTimeScale (movie) * samplesPerFrame
                                / GetMediaTimeScale (media);

        OSStatus err = MovieAudioExtractionBegin (movie, 0, &extractor);

        unsigned long output_layout_size;
        err = MovieAudioExtractionGetPropertyInfo (extractor,
                                                   kQTPropertyClass_MovieAudioExtraction_Audio,
                                                   kQTMovieAudioExtractionAudioPropertyID_AudioChannelLayout,
                                                   0, &output_layout_size, 0);
        if (err != noErr)
            return;

        HeapBlock <AudioChannelLayout> qt_audio_channel_layout;
        qt_audio_channel_layout.calloc (output_layout_size, 1);

        err = MovieAudioExtractionGetProperty (extractor,
                                               kQTPropertyClass_MovieAudioExtraction_Audio,
                                               kQTMovieAudioExtractionAudioPropertyID_AudioChannelLayout,
                                               output_layout_size, qt_audio_channel_layout, 0);

        qt_audio_channel_layout[0].mChannelLayoutTag = kAudioChannelLayoutTag_Stereo;

        err = MovieAudioExtractionSetProperty (extractor,
                                               kQTPropertyClass_MovieAudioExtraction_Audio,
                                               kQTMovieAudioExtractionAudioPropertyID_AudioChannelLayout,
                                               output_layout_size,
                                               qt_audio_channel_layout);

        err = MovieAudioExtractionGetProperty (extractor,
                                               kQTPropertyClass_MovieAudioExtraction_Audio,
                                               kQTMovieAudioExtractionAudioPropertyID_AudioStreamBasicDescription,
                                               sizeof (inputStreamDesc),
                                               &inputStreamDesc, 0);
        if (err != noErr)
            return;

        inputStreamDesc.mFormatFlags = kAudioFormatFlagIsSignedInteger
                                        | kAudioFormatFlagIsPacked
                                        | kAudioFormatFlagsNativeEndian;
        inputStreamDesc.mBitsPerChannel = sizeof (SInt16) * 8;
        inputStreamDesc.mChannelsPerFrame = jmin ((UInt32) 2, inputStreamDesc.mChannelsPerFrame);
        inputStreamDesc.mBytesPerFrame = sizeof (SInt16) * inputStreamDesc.mChannelsPerFrame;
        inputStreamDesc.mBytesPerPacket = inputStreamDesc.mBytesPerFrame;

        err = MovieAudioExtractionSetProperty (extractor,
                                               kQTPropertyClass_MovieAudioExtraction_Audio,
                                               kQTMovieAudioExtractionAudioPropertyID_AudioStreamBasicDescription,
                                               sizeof (inputStreamDesc),
                                               &inputStreamDesc);
        if (err != noErr)
            return;

        Boolean allChannelsDiscrete = false;
        err = MovieAudioExtractionSetProperty (extractor,
                                               kQTPropertyClass_MovieAudioExtraction_Movie,
                                               kQTMovieAudioExtractionMoviePropertyID_AllChannelsDiscrete,
                                               sizeof (allChannelsDiscrete),
                                               &allChannelsDiscrete);

        if (err != noErr)
            return;

        bufferList->mNumberBuffers = 1;
        bufferList->mBuffers[0].mNumberChannels = inputStreamDesc.mChannelsPerFrame;
        //changed  
        bufferList->mBuffers[0].mDataByteSize =  jmax ((UInt32)4096, (samplesPerFrame * inputStreamDesc.mBytesPerFrame) + 16);


        dataBuffer.malloc (bufferList->mBuffers[0].mDataByteSize);
        bufferList->mBuffers[0].mData = dataBuffer;

        sampleRate = inputStreamDesc.mSampleRate;
        bitsPerSample = 16;
        numChannels = inputStreamDesc.mChannelsPerFrame;

        detachThread();
        ok = true;
    }

    ~QTAudioReader()
    {
        if (dataHandle != 0)
            DisposeHandle (dataHandle);

        if (extractor != 0)
        {
            MovieAudioExtractionEnd (extractor);
            extractor = 0;
        }

        checkThreadIsAttached();
        DisposeMovie (movie);

#if JUCE_MAC
        ExitMoviesOnThread ();
#endif
    }

    bool readSamples (int** destSamples, int numDestChannels, int startOffsetInDestBuffer,
                      int64 startSampleInFile, int numSamples)
    {
        checkThreadIsAttached();

        while (numSamples > 0)
        {

            if (lastSampleRead != startSampleInFile)
            {
                    TimeRecord time;
                    time.scale = (TimeScale) inputStreamDesc.mSampleRate;
                    time.base = 0;
                    time.value.hi = 0;
                    time.value.lo = (UInt32) startSampleInFile;

                    OSStatus err = MovieAudioExtractionSetProperty (extractor,
                        kQTPropertyClass_MovieAudioExtraction_Movie,
                        kQTMovieAudioExtractionMoviePropertyID_CurrentTime,
                        sizeof (time), &time);

                    if (err != noErr)
                    return false;
            }



            int framesToDo = bufferList->mBuffers[0].mDataByteSize / inputStreamDesc.mBytesPerFrame;
            bufferList->mBuffers[0].mDataByteSize = inputStreamDesc.mBytesPerFrame * framesToDo;

            UInt32 outFlags = 0;
            UInt32 actualNumFrames = framesToDo;
            OSStatus err = MovieAudioExtractionFillBuffer (extractor, &actualNumFrames, bufferList, &outFlags);

            lastSampleRead = startSampleInFile + actualNumFrames * samplesPerFrame;

            
            
            if (err != noErr)
                return false;

            const int numToDo = actualNumFrames * samplesPerFrame;    // right???

            for (int j = numDestChannels; --j >= 0;)
            {
                if (destSamples[j] != 0)
                {
                    const short* const src = ((const short*) bufferList->mBuffers[0].mData) + j;

                    for (int i = 0; i < numToDo; ++i)
                        destSamples[j][startOffsetInDestBuffer + i] = src [i << 1] << 16;
                }
            }

            startOffsetInDestBuffer += numToDo;
            startSampleInFile += numToDo;
            numSamples -= numToDo;

            if (outFlags & kQTMovieAudioExtractionComplete)
            {
                if (numSamples>0)
                {
                    // fill the rest with zeros
                }
                break;
            };
        }

        detachThread();
        return true;
    }

    

    juce_UseDebuggingNewOperator

    bool ok;

private:
    Movie movie;
    Media media;
    Track track;
    const int trackNum;
    double trackUnitsPerFrame;
    int samplesPerFrame;
    int64 lastSampleRead;
    Thread::ThreadID lastThreadId;
    MovieAudioExtractionRef extractor;
    AudioStreamBasicDescription inputStreamDesc;
    HeapBlock <AudioBufferList> bufferList;
    HeapBlock <char> dataBuffer;
    Handle dataHandle;

    //==============================================================================
    void checkThreadIsAttached()
    {
#if JUCE_MAC
        if (Thread::getCurrentThreadId() != lastThreadId)
            EnterMoviesOnThread (0);
        AttachMovieToCurrentThread (movie);
#endif
    }

    void detachThread()
    {
#if JUCE_MAC
        DetachMovieFromCurrentThread (movie);
#endif
    }

    QTAudioReader (const QTAudioReader&);
    QTAudioReader& operator= (const QTAudioReader&);
};

[jules]

Cool, thanks, that seems to make sense. I’ll check something in soon…

[chkn]

this works for some files great, for other not.
Sometimes the calculated samplesPerFrame ( (int) (GetMediaDecodeDuration (media) / GetMediaSampleCount (media)) )
is 1 sometimes its much higher (~1000), and then we get

I thing you misinterpreted the actualNumFrames from MovieAudioExtractionFillBuffer.
Its the number of sample-frames you are getting, not the number of other (Movie?-)frames…
so we should remove this samplesPerFrame factor.

lastSampleRead = startSampleInFile + actualNumFrames /*deleted * samplesPerFrame*/;
const int samplesReceived = actualNumFrames/*deleted * samplesPerFrame*/;

But at least, this works better, but not perfect, i hear dropouts every second, ( when samplePerFrame is higher )

[jules]

Ah! I didn’t realise it was samples… thanks, I’ll get that checked in shortly!

[chkn]

yes, and

int framesToDo = bufferList->mBuffers[0].mDataByteSize / inputStreamDesc.mBytesPerFrame; ,
should be smaller when the number of requested samples in readSamples (…numSamples) is smaller , to stay in sync ?!