Link with library that contains juce modules

Hi,

I would like to create a static library that includes juce modules (and my own custom code) which a plugin will link with. The plugin should only build the audio_plugin_client modules while all other modules are found in the library. I am using Projucer and have something sort of working. But the plugin also builds all juce modules again because the Projucer requires that as a dependency for the plugin client module. I get no duplicate symbols error though.

Is this actually possible with Projucer? If not, with CMake?

Jelle

I would suggest to just make an additional module that includes all these other modules (and maybe links with other libs, too), instead of a static lib.

JUCE (unfortunately) doesn’t like to link statically and this could also cause very weird problems if the build flags of the static lib are not identical to the flags of the plugin.

The goal is to reduce compilation time.
Using a module would not reduce compilation time because it has to be compiled every time (a library instead can be compiled once and reused).
Compile time flags I have control over so that shouldn’t be an issue.

I think compilation times would be the same, you’d only have to recompile if the modules you’re counting on have changed, which you’d have to do in the static library use case as well as the module use case.

Just make sure your “shared” library is not counting on plugin-related that could change often

Hmm.
If I have multiple plugins that reuse this module, should I only need to build it once in total?

How does the single audio plugin client module “know” that this module file contains all the other modules (because there is a module depenency). What if I go for the static library approach, is there a way to inform the projucer to not add all the module dependencies?

Unfortuantely no, because even the name of the plugin and the version goes into the compile flags. No way around it with the current JUCE design without horrible bugs.

Once you will have even multiple versions of juce_core in each plugin you will suffer some horrible bugs due to multiple versions of the same structs (one with the static lib and one with the plugin) existing in the binary. I know this isn’t how it’s supposed to be, but this is how it is.

Well the plugin name macro is only in the audio plugin client module so in theory I would only need to recompile that module, right?

Unfortunately no, as even things like JUCE_APPLICATION_NAME_STRING are passed right to juce_core, which means now multiple versions of juce_core will be linked to your app.

I agree that it’s a shame, and JUCE needs to offer a better way to do these things. If you use a dll, however, you won’t suffer from those problems because then each call will have its own version of JUCE to link against.

Isn’t JUCE_APPLICATION_NAME_STRING only for standalone apps?

If you link multiple versions of core with the app, why doesn’t it throw some multiple symbols error?

At least in my tests, it did throw that, and in other cases it just caused weird issues at runtime (because the linker might think it resolved function calls but did so with the wrong version of the calls).

Yes but it’s defined in a plugin too.
The problem is, even if a specific flag is not used in a plugin, the compiler still passes that so it’s a different binary.

My goal is to optimize compilation time so I don’t want to compile any juce module code twice because that would not reduce the compilation time. Also I’m aware it can cause weird bugs if you have duplicate symbols for functions shared over multiple translation units.

However, I am not fully convinced yet that this is impossible with your reasoning about the preprocessor definitions passed into core module.
For instance JUCE_APPLICATION_NAME_STRING is as far as I know only used for CMake projects so it does not exist in my project since I am using Projucer. The Projucer in the plugin project generates a JucePluginDefines.h file which defines a bunch of plug-in specific variables such as JucePlugin_Name, JucePlugin_ManufacturerCode, or JucePlugin_IsSynth. These are only used in the juce_audio_plugin_client module, not in any other. That would mean that all plug-ins can use pre-compiled versions of all modules except for the juce_audio_plugin_client module.

As long as I make sure each plugin compiles this juce_audio_plugin_client module, it can use all the other pre-compiled modules and that should work fine. In my case all the plug-ins will compile with the same preprocessor flags (except for the plugin name and code obviously).

I might be completely wrong and this approach could be risky, so I would like to know what the risks are. I just don’t want to give up immediately.

The only thing I can not figure out is how to setup my Projucer Plugin project in a way that it skips the compilation of all the juce modules (because otherwise there is no reduction in compilation time and I’m not solving any problem at all). I just want to compile the audio_plugin_client module. Is there for instance some flag that I can explicitly set that will make the build system skip compiling some modules?

The module system is a bit misleading in that sense.
When you add, say, juce_core to your project, what ends up under the hood is that the projucer/CMake will actually tell the build system that juce_core.cpp is a part of your project.

Then, it builds juce_core.cpp with all the build flags that you defined in your project, regardless if the actual code uses those flags.

Now, lets say you’re linking your project with some other static library that has juce_core.cpp in it, and there’s some non-inlined function call like a function from juce::String.

When your static lib gets built, the binary will have some note in it saying where to find all functions related to juce_core (which have already been compiled!)
When your plugin gets built (with yet another version of juce_core), notes referring to functions with the exact same name will show up.

Then when the linker comes in, it would find multiple versions of the exact same functions, and either warn you about it, or “fix that” behind the scenes by removing one, which might be the wrong one. When optimizers and LTO comes in, chaos will very likely endure.

The “Correct” solution would be to compile juce_core itself as a library, and then link it with both the plugin and your static lib. But unfortunately JUCE doesn’t support that at the moment. It will work if you’re ok with creating a dll though, but that has other complications.

1 Like

Thank you Eyal for explaining. I understand how this works.

So the Projucer generates an include_juce_core.mm (on mac) file, which includes juce_core.cpp and so it gets compiled. I understand that I’m stuck with 2 versions of core.cpp then.
But, my question is if there is a (hacky?) way to tell the build system to skip it, even though it’s already added to the project.

I haven’t tested this, but in cmake you could try:

add_library (moduleLib STATIC)

target_link_libraries (moduleLib PUBLIC juce::juce_core)

juce_add_plugin (myPlugin ...)

target_link_libraries (myPlugin PUBLIC moduleLib)

If you use CMake, yes that could work.

Unfortunately, that won’t work even if you do use CMake, you need to configure the library with juce_add_xxx or _juce_initialise_target() and even then it won’t link correctly with the plugin without setting some extra flags.

@reuk can probably explain the mechanics better. But yes, that’s how I’d hope JUCE would allow us to work.

That is unfortunate.

The way I’ve structured my own codebase is to factor out all code without JUCE dependencies into plain C++ libraries, and then create a “JUCE layer” - a JUCE module that depends on the static library and contains adapters for the JUCE API. The JUCE layer will be recompiled every time, but the static library will not. You can try to move as much of your code as possible out of the JUCE layer.

That sounds like a nice approach @benvining.

For the Projucer things are a bit different. I know everyone is moving to CMake and I probably should, but I would like to get this working with Projucer. The Projucer has this “module dependency” system which prevents any JUCE project from containing any modules without their dependencies. This is a great system, except that there is no way of enforcing a missing dependency, which I really would like to do giving that my modules are in the static library. I would like to only build the audio_plugin_client module and nothing else.

My request to the JUCE team, @reuk, is if it would be possible to change the Projucer dependency error (“at least one of your modules has missing dependencies”) to a warning that you can ignore. I think it’s really nice that the Projucer is hinting you but a warning would do the job and the user can decide to ignore it. At least allow the command line --resave to ignore this error. Something like “Projucer --resave --ignore-error” or “Projucer --resave --allow-missing-dependencies”.

You don’t want to allow modules to ignore/exclude their dependencies, what you’re really after is allowing a module to be built as a library and then linked against instead of recompiled as part of the consuming project.

This is also a feature request I’d support. But in the meantime, I would suggest not trying to fight or hack the JUCE module system – each module needs its dependencies present in the project, and there’s really no way to get around this without major rewrites to the JUCE module system infrastructure.