Number of channels to use to process AudioBuffer from a modulating parameter

I am working on a plugin where you can control the parameter of a node(AudioProcessor) from the output of another node, my question is while processing for the next node would it be better if we modulated both channels respectively based on the input channels, or we use only one channel like only one side or the average etc… to modulate? would having both the channels modulate mess up the audio for a longer effects chain?