Windows: Plugin + dynamic linked library + dynamic runtime – How to do it right?

After having successfully sticked to statically linked libraries including statically linked runtime library for all my Windows plugin projects until now, I’m now having to use a third party library that is only available as dll and which is built against a dynamically linked runtime library.

While I have a clear idea what static vs. dynamic linkage means technically, I’m not so sure about how to properly deploy a plugin along with dlls it depends on, doing 99% of my work on Mac OS, I’m lacking some knowledge of what’s the right way to organize all this on windows.

First question: Where to put the external library?

If both, vst3 and aax versions of the plugin need the library, where should it live? I found this resource on the order of where dlls are searched. However, it is specific to applications. It says that it first looks in “The directory from which the application loaded.”. In case of a plugin, does this translate to “next to the plugins dll”? If this works, is that recommended (as it would basically mean that I had to copy it twice for each plugin format)? Or is a system location recommended? But if so, how is it ensured that no other installer would overwrite the library with another version of it or whatnot?

Second question: How to handle the Visual C++ redistributable?

In this help article microsoft describes the differences between Central Deployment and Local Deployment. I get the feeling that Central Deployment is the way to go…? As I don’t use a msi based installer but an Inno Setup based one, I guess I have to use the Redistributable package exe, right? Are there any examples on using that along with Inno?

2 Likes

Hey @PluginPenguin! I’m currently attempting the same thing. Did you find the best solution to this? In my situation I cannot build a static library and need to dynamically link to a pre-built DLL.

So far I have came across using https://learn.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-setdlldirectorya
but, some people feel this isn’t ideal as you are adding a library path to the host application. I feel this is the best I can come up with however.

Any input from your experience would be amazing, thank you!

I wrote an experimental spatializer plug-in using Steam Audio and got it working on Windows by loading the library dynamically at runtime. I’m a bit rusty/fuzzy on the process so I may not remember everything or get it right, but I’ll offer the insights I did have.

First, I can’t really answer the question about the VC++ redistributable, since I did not need it in my case.

The biggest issue I ran into, is that in order to automatically load and use a dynamically link to a .dll, the .dll either has to exist in a path that Windows will scan for automatically, or in the same path as the executable that is loading it (in the same location as the DAW executable).

I didn’t want to install my plug-in’s needed .dll files to something like C:\Windows\System32, and it’s obviously not feasible to install the plug-in into the DAW’s *.exe folder (let alone all possible DAWs), so the only choice I really had was to load it manually from the plug-in instance itself at runtime from a known location.

In my situation, the Steam Audio library is distributed as a .dll binary, and matching header file. Since I needed to manually load the .dll at runtime, and my solution was to create my own wrapper class around their API. Fortunately, their header file had very consistent syntax, so I was able to write a script that generated most of my wrapper file’s source code for me, otherwise it would have been pretty tedious (my script wrote most of the .h and .cpp to wrap their methods for me, but you can do it by hand).

The 3rd Party API

For example, in their included header file for the Steam Audio API, they might have something like this:

/** Creates a context object. A context must be created before creating any other API objects.

    \param  settings    Pointer to the `IPLContextSettings` struct that specifies context creation parameters.
    \param  context     [out] Handle to the created context object.

    \return Status code indicating whether or not the operation succeeded.
*/
typedef IPLerror(IPLCALL* iplContextCreate_api)(IPLContextSettings* settings, IPLContext* context);

Wrapper Class

I created a wrapper class called SteamAudioAPI, in it, I include their original header file with all of their typedefs, structs, enums, etc… Next for each call in their original API, I create a private declaration of the function being wrapped:

private:
    iplContextCreate_api iplContextCreate_func;

And then a public wrapper method on my wrapper class:

public:
    IPLerror iplContextCreate(IPLContextSettings* settings, IPLContext* context);

Loading and Binding at Runtime

As I mentioned earlier, I have to manually load and bind phonon.dll at runtime from the JUCE plug-in instance. So my SteamAudioAPI class has methods for this:

/* Gets whether or not the dynamic library has been loaded into memory.*/
bool isLibraryLoaded() { return isLoaded; }

/** Sets the location of the Cymatic Somatic Core library file to load.
*   @param path The absolute path to the DLL file to load
*/
void setLibraryPath(juce::String path);

/** Loads the Steam Audio library into memory.
*	@param modulePath	The file path to the Steam Audio dynamic library.
*/
void load(juce::String modulePath = {});

/* Unloads the Steam Audio library from memory.*/
void unload();

My implementation looks like this:

#include "SteamAudioAPI.h"

#ifdef JUCE_WINDOWS
#include <windows.h>
typedef LPCWSTR OS_STRING;
#else
//typedef std::String OS_STRING;
#endif

namespace CymaticSomatics
{
#ifdef JUCE_WINDOWS
	HMODULE steamAudioModule;
	LPCWSTR steamAudioModulePath = L"phonon.dll";
#endif

	void SteamAudioAPI::setLibraryPath(juce::String path)
	{
#ifdef JUCE_WINDOWS
		steamAudioModulePath = path.toUTF16();
#endif
	}

	void SteamAudioAPI::load(juce::String modulePath)
	{
		isLoaded = false;
#ifdef JUCE_WINDOWS
		if (modulePath.isNotEmpty()) steamAudioModulePath = modulePath.toUTF16();
		steamAudioModule = LoadLibraryW(steamAudioModulePath);
#endif
		if (!steamAudioModule || !bindToAPI()) return;
		isLoaded = true;
	}

	void SteamAudioAPI::unload()
	{
		if (!isLoaded) return;

#ifdef JUCE_WINDOWS
		FreeLibrary(steamAudioModule);
#endif
	}

Loading Manually

Specifically, to load a .dll at runtime, use the Windows system call LoadLibraryW()

Unloading Manually

To unload the library manually, use the Windows system call FreeLibrary()

Binding Methods to their Wrappers

OK, so that loads or unloads the .dll at runtime, but how do you actually bind to the methods and call them?

My wrapper class also has a private method bindToAPI() which is called from the load() method:

bool bindToAPI();

It’s implementation looks like this:

bool SteamAudioAPI::bindToAPI()
{
#ifdef JUCE_WINDOWS
    iplContextCreate_func = reinterpret_cast<iplContextCreate_api>(GetProcAddress(steamAudioModule, "iplContextCreate"));
    if (!iplContextCreate_func) return false;

    iplContextRetain_func = reinterpret_cast<iplContextRetain_api>(GetProcAddress(steamAudioModule, "iplContextRetain"));
    if (!iplContextRetain_func) return false;

    iplContextRelease_func = reinterpret_cast<iplContextRelease_api>(GetProcAddress(steamAudioModule, "iplContextRelease"));
    if (!iplContextRelease_func) return false;

    /* It's quite long, so I've omitted the rest, but the idea is that if a single
       binding fails, the whole binding operation is considered a failure.
    */

#endif

    return true;
}

After this method returns true, then all of the publicly exposed methods of the SteamAudioAPI are now bound to the methods in the manually loaded .dll, and just wrap/forward calls to them.

AudioProcessor

In my AudioProcessor class, I have an instance of the SteamAudioAPI class:

private:
    // The local Steam Audio API module
    SteamAudioAPI steamAudio;

I wrote this as a JUCE module, so I can include it in any JUCE project, and since the module references the original header provided by Steam, all of the basic types are defined so I can work with them directly from the AudioProcessor code.

I manually install the Steam Audio .dll into the same location as my VST3 since I can control that, and have a few utility methods to resolve the path:

    // Get the parent directory the processor is executing from
    juce::String getPluginDirectoryPath()
    {
        return File::getSpecialLocation(File::SpecialLocationType::currentExecutableFile).getParentDirectory().getFullPathName();
    }

    // Gets the Steam Audio dynamic library binary file path
    juce::String getSteamAudioPath()
    {
        return getPluginDirectoryPath() + File::getSeparatorChar() + "phonon.dll";
    }

I also have some methods on the AudioProcessor that load/release the API at runtime:

    // Creates and initializes the Steam Audio engine
    void createSteamAudio();

    // Releases the Steam Audio engine
    void releaseSteamAudio();

I call createSteamAudio() in the AudioProcessor’s constructor, and call releaseSteamAudio() in the AudioProcessor’s destructor.

Here’s what their implementation looks like:

void CySoSpatializer::createSteamAudio()
{
    auto steamAudioFilePath = getSteamAudioPath();
    DBG("Loading Steam Audio from: " + steamAudioFilePath);
    steamAudio.load(steamAudioFilePath);

    if (!steamAudio.isLibraryLoaded())
    {
        errorLoadingSteamAudio = true;
        DBG("Error loading library");
        return;
    }
    else
    {
        contextSettings.version = STEAMAUDIO_VERSION;
        auto error = steamAudio.iplContextCreate(&contextSettings, &context);
        hrtfSettings.type = IPL_HRTFTYPE_DEFAULT;
    }
}

void CySoSpatializer::releaseSteamAudio()
{
    if (!errorLoadingSteamAudio)
    {
        steamAudio.iplContextRelease(&context);
    }
}

That stuff will obviously be specific to the library you are working with, but the main point was to show how I call steamAudio.load() with the path of the .dll, which loads, and binds the API - and the check to make sure it loaded correctly.

Conclusion

I’m not sure ultimately how much this applies to your scenario, but I know I banged my head on this until I figured out a valid solution, so hopefully you can get some insight out of it.

1 Like

CHOC has a few utilities that might be useful: choc/platform at main · Tracktion/choc · GitHub

1 Like

@Fyfe93 thanks for bringing up that question again – it’s always a nice to share the solution with the community after asking for help :wink:

We ended up installing our .dll files in Windows\System32. However, we prefixed them with our company name, so that we can be pretty sure that no other installer will overwrite them with an unexpected version. This works quite reliable, I cannot remember any issue with this since the release of our first plugin that used these libraries, which was probably 2 years ago or so. I actually don’t really remember the background of the second part of my question regarding the redistributable and I cannot remember any real-world problems in that context, so it might not be a real issue :smiley:

The approach that @Cymatic shared is a valid alternative in some cases. In particular, it will work great in case your library exposes a pure C interface – loading mangled C++ symbols at runtime is quite tricky and usually not recommended. Also, if other third party code accesses the library, this won’t work without modifying that code to use your runtime loaded symbols. In any case, I’d propose keeping it portable and use the juce::DynamicLibrary class to handle all that function loading in a cross-platform way.

Last but not least, Jules’ chock::memory:: MemoryDLL really caught my attention when listening to his talk at the last ADC, unfortunately I haven’t found the time to take a closer look at it. Still it basically has the same constraint as loading the library from the file system at runtime mentioned above.

1 Like

Hey @PluginPenguin @olilarkin @Cymatic , Thank you for all your input on this. Definitely a difficult problem with no perfect solution it seems. In my scenario, the opening of the DLL at runtime and function loading stuff won’t work as I need to instantiate member objects that are used throughout my project. I am also using multiple DLLs that are all interacting with each other so I imagine it being difficult to get the mapping right when creating all the handles I would need to use.

I think installing the DLLs to Windows\System32 with a unique prefix seems to be the way to go for my situation also. I’ll give it a shot and report back on whether it worked or not! Thank you for your help! :blush:

1 Like

Hi @Fyfe93 , I talked to you on Discord too and guess you’re referring to Neutone. So for a public library like pytorch, how can we add unique prefixes while still linking it correctly?

Hey @200gaga ! So I think the solution will be to build libtorch from source and to edit the build scripts to use a custom naming of the libraries so that the DLLs can find/link to each other correctly. There might be a way of changing the names of the pre-built DLLs that PyTorch supply manually by using the equivalent of install_name_tool on Mac but for Windows. Unsure what tool to use for that? If someone can share that info that would be great!

I would probably prefix all the library names with neutone_versionid in this case which would prevent any future conflicts.

Thanks! @Fyfe93 . It would be great if you could share the process of build on Windows here.

@PluginPenguin since you are using Inno Setup, you can use it as a bootstrap for installing the redistributable as well, it can also detect if it is already installed so that it only installs it as part of the installation process if necessary (there are registry entries for this).

I recall using Inno Setup’s scripting (Pascal IIRC, oldschool!) for some of the needed logic

I’ve been able to load a DLL from a custom location using the /DELAYLOAD linker option which I don’t think has been mentioned here. I’ve only tested this locally so far. I’d be interested to know if anyone spots a problem with this solution.

Usually implicitly linked DLL’s will be loaded at the same time as the executable that uses it. With delayed linking the library won’t be loaded until the first call to a function in the DLL. It also allows you to customise how the library is loaded. I’ve used a hook to load my library from a custom location.

Reference - Linker support for delay-loaded DLLs | Microsoft Learn

The build setup in CMake will look something like this

juce_add_plugin (MyPlugin ...)

set (myLibDirectoryPath ...)

target_link_libraries (MyPlugin
    PRIVATE
        delayimp
        ${myLibDirectoryPath}/MyLib.dll.lib)

target_link_options (MyPlugin PUBLIC "/DELAYLOAD:MyLib.dll")

add_custom_command (TARGET MyPlugin_VST3 PRE_BUILD
    COMMAND
        ${CMAKE_COMMAND} -E copy
        "${myLibDirectoryPath}/MyLib.dll"
        "$<TARGET_FILE_DIR:MyPlugin_VST3>/../Resources/MyLib.dll"
    COMMENT "Copying MyLib.dll to MyPlugin.vst3 Resources directory...")

delayimp is Microsoft’s default implementation for delayed loading. You can provide our own implementation if desired, but I found it simpler to stick with the default implementation and use hooks to customise it. Somewhere in your C++ source you can write a hook that loads the DLL.

#include <delayimp.h>
#include <libloaderapi.h>

juce::File getResourcesDirectory()
{
    return juce::File::getSpecialLocation (juce::File::SpecialLocationType::currentExecutableFile).getChildFile ("../../Resources");
}

FARPROC WINAPI delayHook (unsigned dliNotify, PDelayLoadInfo info)
{
    const auto libName = "MyLib.dll";

    if (dliNotify == dliNotePreLoadLibrary && juce::String (info->szDll) == libName)
    {
        const auto libPath = getResourcesDirectory().getChildFile (libName);
        auto loadedLib = LoadLibraryA (libPath.getFullPathName().toRawUTF8());
        jassert (loadedLib != nullptr);
        return FARPROC (loadedLib);
    }

    return nullptr;
}

ExternC const PfnDliHook __pfnDliNotifyHook2 = delayHook;
2 Likes