@Fyfe93 thanks for bringing up that question again – it’s always a nice to share the solution with the community after asking for help 
We ended up installing our .dll files in Windows\System32. However, we prefixed them with our company name, so that we can be pretty sure that no other installer will overwrite them with an unexpected version. This works quite reliable, I cannot remember any issue with this since the release of our first plugin that used these libraries, which was probably 2 years ago or so. I actually don’t really remember the background of the second part of my question regarding the redistributable and I cannot remember any real-world problems in that context, so it might not be a real issue 
The approach that @Cymatic shared is a valid alternative in some cases. In particular, it will work great in case your library exposes a pure C interface – loading mangled C++ symbols at runtime is quite tricky and usually not recommended. Also, if other third party code accesses the library, this won’t work without modifying that code to use your runtime loaded symbols. In any case, I’d propose keeping it portable and use the juce::DynamicLibrary class to handle all that function loading in a cross-platform way.
Last but not least, Jules’ chock::memory:: MemoryDLL really caught my attention when listening to his talk at the last ADC, unfortunately I haven’t found the time to take a closer look at it. Still it basically has the same constraint as loading the library from the file system at runtime mentioned above.