Problem selecting input/output devices in Low Latency Mode due to sample rate OR buffer size mismatch

I’m working on a Windows (10) standalone app doing some audio processing and stumbled upon some problems with the selection of my audio inputs/outputs.

Following what has been said in previous discussions, this issue happens when using the Low Latency Mode.
Mainly the issue has to do with the impossibility of selecting my input/output due to sample rate mismatch (this has indeed been discussed many a time on the forum) or to buffer size mismatch.

The sample rate problem was discussed in previous threads (e.g. here Enabling Windows Audio's SRC where the author mention many other threads bringing this problem up) and the thread describes a patch to the problem but this doesn’t seem to work on my machine (?!?). See, e.g. this gif where I try to select the internal mic/speakers in Windows Audio (Low Latency Mode) in the DemoRunner.exe and how I can’t select them simultaneously since they have different sample rate. Only after changing manually the sample rate of one of the devices (the mic) from the Windows Sound Panel, am I able to select them simultaneously.

I also realised that the app behavior might depend upon which audio driver my laptop is running (as another comment Enabling Windows Audio's SRC - #26 by sonosaurus in the same thread above pointed out). In the previous example, where I show how I could manage to get around the sample rate problem I was using the High Definition Audio Driver from Microsoft. If, however, I use the default driver installed in my machine (Realtek Audio Driver), things get suddenly hopeless as the next gif shows. Basically, the problem here is that the two devices do share a common sample rate, but do not share a common buffer size (the speaker only supports 512 while the mic only supports 480) and, unlike the sample rate, this option doesn’t seem to be changeable in the Windows Sound Panel…

Any idea of what one could do in this situation?
I’d stick to simple Windows Audio if I could, but my app doesn’t really work properly with the buffer sizes that I’m planning to use (256-512).