Ableton Live 11 fails to resolve rpath of dylib in VST3 plug-in, but other DAWs work

1

I have a VST3 plug-in that uses a precompiled dynamic C library (libphonon). On Windows, I put the .dll in C:\Windows\system32, and it works as expected in all hosts/DAWs that I’ve tried. Unfortunately, I don’t have access to the source of the dynamic library.

On macOS, I place the dylib in /usr/local/lib, and it works fine on all hosts that I’ve tried except for Ableton Live 11. The machine I’m testing on has an M1 processor/Apple silicon. I’m compiling against JUCE 7.0.4. When I don’t link with the dylib, the plug-in does show up in Ableton, so it’s definitely something about linking with it.

Using otool -L on the dylib, I can see the first entry is:

@rpath/libphonon.dylib

I added debug entitlements to Ableton, and was able to get a bit more insight when I launched it from Xcode. Ableton’s debug output will sometimes say that @rpath/libphonon.dylib failed to load/not found, which explains why the plug-in fails Ableton’s scanning.

I’ve also tried using install_name_tool -change to change @rpath/libphonon.dylib to the absolute path /usr/local/lib/libphonon.dylib on the VST3 (MyPlugin.vst3/Contents/MacOS/MyPlugin) without any luck.

Alternatively, I’ve also tried setting the VST3 to @loader_path/../Frameworks/libphonon.dylib using install_name_tool, and then adding libphonon.dylib to MyPlugin.vst3/Contents/Frameworks/libphonon.dylib, in an attempt to bundle it with the plug-in (and removing /usr/local/lib/libphonon.dylib). When I do this, if I launch the DAW from Xcode via the debugger (AudioPluginHost, Reaper, Tracktion Waveform, etc., not Ableton though), the library resolves, but if I just open the app without a debugger attached they all crash for some reason.

So essentially, copying the dylib to /usr/local/lib is working great, except Ableton, and my attempts to use install_name_tool to adjust the VST3’s library is showing mixed results and still no love in Ableton.

#1, does anyone know what or why Ableton is doing something different with dynamic library paths on macOS vs. other DAWs?

#2, does anyone know the best way to include dynamic libraries with plug-ins on macOS that works across the board, including in Ableton Live?

Curious if anyone else has run into this behavior.

Update:

This is what the Ableton debug output says:

2023-10-05 14:27:22.954629-0700 Ableton Plugin Scanner[789:11485] Error loading /Users/michael/Library/Audio/Plug-Ins/VST3/MyPlugin.vst3/Contents/MacOS/MyPlugin:  dlopen(/Users/michael/Library/Audio/Plug-Ins/VST3/MyPlugin.vst3/Contents/MacOS/MyPlugin, 262): Library not loaded: @rpath/libphonon.dylib
  Referenced from: /Users/michael/Library/Audio/Plug-Ins/VST3/MyPlugin.vst3/Contents/MacOS/MyPlugin
  Reason: image not found
error: Failed to load plugin: The bundle “My Plugin” couldn’t be loaded because it is damaged or missing necessary resources.
2

on OSX, the safest way is to put the dylib in your bundle and fix the path to it in your exe

function(fixOSXBundle targetName)
 add_custom_command(
          TARGET ${targetName} POST_BUILD
          COMMAND
            ${CMAKE_COMMAND} -E copy
            "/usr/local/lib/libphonon.dylib"
            "$<TARGET_FILE_DIR:${targetName}>/../Frameworks/libphonon.dylib"
          )
        add_custom_command(
          TARGET ${targetName} POST_BUILD
          COMMAND install_name_tool -change @rpath/libphonon.dylib @loader_path/../Frameworks/libphonon.dylib $<TARGET_FILE:${targetName}>
          )
    endfunction()

and call
fixOSXBundle(${PROJECT_NAME}_VST3)
and your other different target

1 Like
3

Thanks for the reply. I’m not using cmake, but I can discern the steps from the code you shared.

I might not have been clear in my previous post, but I’ve essentially done the steps you’ve outlined manually:

  • copying the dylib into the VST3 bundle in a Frameworks directory
  • using installl_name_tool to change the relative path to the @loader_path path within the bundle on the VST3 binary in the MacOS directory
  • verifying the library path change using otool -L and otool -L on the VST3 binary

However, if I attempt to load the VST3 into a DAW/host after those steps, I observe the following:

  • The host will crash when attempting to load the VST3 (from say a demo project that has one instance of the VST3 in it)
  • When launching the host via Xcode debugger attached session, projects with the VST3 saved in them will load ok via the @loader_path change. Why would they crash in a non-debugger-attached process, vs. a debugger-attached-process - that seems weird to me, and I can verify it is loading from the bundle when debugging.
  • After the changes of moving it to the VST3 bundle as you’ve outlined, the VST3 will no longer scan in hosts (debugger-attached, or not - it basically crashes on scan, something relating to the rpath change apparently).

On top of that, is still doesn’t show up in Ableton, so I gain nothing from leaving it as-is and placing it in /usr/local/lib, and it actually stops working in all hosts.

I’m definitely new to hacking dynamic library paths, and I feel like I must be missing something. Would there be a difference to performing those two steps you’ve outlined in a post build step vs. doing it manually after the VST3 is built as I have done? Because if not, that cmake function would actually make my situation worse, as outlined by the result of applying the steps manually.

It also still leaves me completely unsure what Ableton might be doing differently when loading dynamic libraries that would cause the @rpath/libphonon.dylib + /usr/local/lib location to work in everything except Ableton.

4

Well, I answered my own question: applying the change via install_name_tool after the build within the bundle did not work. Adding an rpath via linker flags instead was the answer.

In Projucer’s Extra Linker Flags in the Xcode exporter: -rpath @loader_path/../Frameworks

Working correctly now in all hosts.