Performing separate convolutions on Left and Right Channels for 3d Audio HRIR

I have been trying to perform convolution using Head Related Transform Functions(HRTF), or more exactly using the Head Related Impulse Responses(HRIR) from Cipic. I have successfully convolved the entire buffer L/R or Channels 1 and 2, but what I need to do is perform one convolution on the left Channel, and then a separate convolution on the right channel to create the effect of 3d audio.

This is where I am performing the convolution right now. Is this the right spot in the signal flow chain? How to I separate and return the bufferToFill correctly if I had two separate convolution processors? Should this be done in the processBlock function instead? Any help would be appreciated.

 void getNextAudioBlock (const AudioSourceChannelInfo& bufferToFill) override
    {

        if (readerSource.get() == nullptr)
        {
            bufferToFill.clearActiveBufferRegion();
            return;
        }
        transportSource.getNextAudioBlock (bufferToFill);
        conProcessorLeft.processBlock(*bufferToFill.buffer, emptyMidi);

    }