JUCE module dependencies with version ranges

Hi there!
We are currently trying to separate our codebase that is shared across plugins in order to be a more flexible when updating one or another part of the plugins functionality.
As of now, we have one big repository that hosts a couple of JUCE modules that is added to the plugins via git submodules. The idea now would be to split the big repository into a couple of new ones, each just hosting one single JUCE module. This means that a plugin would have a couple of submodules instead of one.

In principle this already works fine, but I can see problems coming up in the future, namely when dealing with inter-module dependencies. While simple dependency management is already possible with the current JUCE module format, it can only handle make sure that another module is present.

The approach that I’d imagine however would require the module system to make sure that another module is present with a given version, or maybe even a version range. This would probably require a change to the module format, I’d propose something along the following lines:

...
 dependencies:     juce_audio_basics@6.0.1, juce_audio_formats@<7.0.0, juce_events@[6.0.2..6.1.0], juce_gui_basics@>5.4.1
...

If a dependency isn’t met, then a compilation error could arise, telling the programmer to update a certain module. I think that this could be a very useful addition to the module format.
Do you think that this is feasable?

The topic you’re describing is true, but I don’t think it’s really related to the JUCE modules.

If you use a package manger, such as Conan, VCPKG or CPM.cmake you can set up your dependency tree there including versioning.

Of course i thought of conan.io, and we already use it extensively during development. I can imagine that for modules that don’t change very often this would be a viable option, since it might be possible to pre-compile individual modules into binaries for them to be handled via one of the package/dependency managers out there.
But the development workflow with custom JUCE modules is somewhat different, since we’re dealing with source code here. I really don’t want to handle source code (that is managed via git already) with another package manager. On the other hand, i really don’t want to handle source code just via the package manager and avoid git.

The fact that the source code is available directly via the submodule, and that small changes in the modules can immediately be observed is very convenient, and I don’t think that we’ll be able to move away from this workflow.

1 Like

JUCE modules (unfortunately) can’t be precompiled into binaries, even though that would have been a great property.

The reason is: Currently JUCE needs all the modules to be build with your app/plugin build flags. So the way it works now, the modules get added as source files to your project and get rebuild with each project.

So you can’t distribute modules as binaries even when all of them would share the same version.

Package managers like CPM.cmake (and surely Conan too?) can also grab the matching sources just like you do with gitsubmodules.

You can see an example of how I use it here:

In this case this is just linking with the JUCE develop branch, but in my code I have multiple repos with custom modules linked in that way, including versioning requirements, specific commits, etc.

The advantage is that this way of doing it is ‘viral’, so the knowledge of required versions will get to dependencies of dependencies, etc, and the package manager will be able to resolve a more complicated dependency graph and fetch the correct thing from git.

I haven’t tried it yet, but i’d imagine that when developing own modules based on the JUCE module structure it would still be possible to pre-build and package them- they’d be just static binaries after all, right? Since you’re in control in this scenario, you can avoid relying on some compiler flags (like Juce_PluginName or sth similar). In my mind it should be possible to create one big static library from the whole JUCE codebase including all modules, which is then linked against the custom modules. But this is actually a different topic :slight_smile:

Thank you for the suggestion for CPM.cmake, I only now understood how it is intended to be used, it looks very promising. Am i reading this correctly in that it will clone the repository in question into a local folder? E.g. would i be able to check out a branch in the repository downloaded by CPM and make some modifications right away?

Do you have any example how you would have more complicated versioning requirements, e.g. not only relying on a tag but rather CPMAddPackage("gh:catchorg/Catch2@<2.5.0")?

How does CPM handle nested dependencies?
e.g. imagine this situation

product/               -> requires JUCE > 6.0.0
  ext/
    JUCE/
    MyModule1          -> requires JUCE > 6.0.1
    MyModule2          -> requires JUCE > 6.1.0

If i were to move to the CPM approach, i’d create git repositories for MyModule1 and MyModule2, and
in their CMakeLists.txt i’d add something like the following, as in your example:

CPMAddPackage(NAME JUCE GITHUB_REPOSITORY juce-framework/JUCE GIT_TAG origin/6.0.1)

Is CPM clever enough to notice that both MyModule1 and MyModule2 need JUCE, but with different versions?

Unfortunately no. You can of course do that with your own code if it’s not based on JUCE, but once you include any JUCE module (even juce_core) you will have to recompile the module (and all dependency modules) along with your specific app flags.

I agree that it’s not so great, and as someone who manages many JUCE based projects it would be amazing if that changes and a static library would be possible, but as of now and the near future I would imagine it won’t be the case.

Yes. The great quality is let’s say you have many interconnected modules but want to be maintaining (change) one, you can use:

cmake -G Xcode -B build -D CPM_Toolbox_SOURCE =path/to/toolbox

And then all nested repositories would know to look at your local folder instead of git. That’s especially important when you’re maintaining a dependency of a dependency, which isn’t something git submodules know how to handle.

CPM does handle versioning requirements in each dependency, there’s a VERSION argument when calling it - check out the readme on the github page for example.

Yes, but… it will only really work in a “forward” way. So if you have a module that requires JUCE 5 and a module that requires JUCE 6, you can’t have each module build with it’s own version.

In that case the package manager will “resolve” the dependency requirements and will only clone the very latest version needed, so your modules that required the old versions might not compile if the API changed between those versions.

1 Like

Seems like a very nice way of handling non-binary dependencies, i’m definitely going to play around with this a bit! Thank you very much for your input!

And still… if JUCE would be able to handle module dependencies via the module headers, all this external tooling would be kind of obsolete… @reuk do you think that this would be a possibility, is there any interest of integrating such a system into JUCE?