I think big gains (x2) could be achieved by using the parallel property of the IIR multi-phase filters. It’s two independent filters in parallel outputting interleaved samples, so they can be calculated at the same time using SIMD. The FIR filter could also be done using SIMD and could scale even more than 2-fold as it’s parallel by design. This would all apply to single-channel oversampling and by extension also multi-channel.
Of course, the internal processing loop is the most important thing to optimize, but the oversampling itself is also oversampled for higher ratios and the filters are executed many times. For instants, an 8x oversampler filters the signal 6 times through all the stages and the innermost are running at 4x the sample rate. So overall we end up filtering 2*(4+2+1)x = 14x the amount of incoming audio samples.
btw. I had a thought about the -75dB max quality… would you use an audio interface with 75dB SNR? In my opinion, the same quality demands apply because the oversampling should be transparent to the audio.