JUCE module dependencies with version ranges

Feature Requests
1

Hi there!
We are currently trying to separate our codebase that is shared across plugins in order to be a more flexible when updating one or another part of the plugins functionality.
As of now, we have one big repository that hosts a couple of JUCE modules that is added to the plugins via git submodules. The idea now would be to split the big repository into a couple of new ones, each just hosting one single JUCE module. This means that a plugin would have a couple of submodules instead of one.

In principle this already works fine, but I can see problems coming up in the future, namely when dealing with inter-module dependencies. While simple dependency management is already possible with the current JUCE module format, it can only handle make sure that another module is present.

The approach that Iā€™d imagine however would require the module system to make sure that another module is present with a given version, or maybe even a version range. This would probably require a change to the module format, Iā€™d propose something along the following lines:

...
 dependencies:     juce_audio_basics@6.0.1, juce_audio_formats@<7.0.0, juce_events@[6.0.2..6.1.0], juce_gui_basics@>5.4.1
...

If a dependency isnā€™t met, then a compilation error could arise, telling the programmer to update a certain module. I think that this could be a very useful addition to the module format.
Do you think that this is feasable?

2

The topic youā€™re describing is true, but I donā€™t think itā€™s really related to the JUCE modules.

If you use a package manger, such as Conan, VCPKG or CPM.cmake you can set up your dependency tree there including versioning.

3

Of course i thought of conan.io, and we already use it extensively during development. I can imagine that for modules that donā€™t change very often this would be a viable option, since it might be possible to pre-compile individual modules into binaries for them to be handled via one of the package/dependency managers out there.
But the development workflow with custom JUCE modules is somewhat different, since weā€™re dealing with source code here. I really donā€™t want to handle source code (that is managed via git already) with another package manager. On the other hand, i really donā€™t want to handle source code just via the package manager and avoid git.

The fact that the source code is available directly via the submodule, and that small changes in the modules can immediately be observed is very convenient, and I donā€™t think that weā€™ll be able to move away from this workflow.

1 Like
4

JUCE modules (unfortunately) canā€™t be precompiled into binaries, even though that would have been a great property.

The reason is: Currently JUCE needs all the modules to be build with your app/plugin build flags. So the way it works now, the modules get added as source files to your project and get rebuild with each project.

So you canā€™t distribute modules as binaries even when all of them would share the same version.

Package managers like CPM.cmake (and surely Conan too?) can also grab the matching sources just like you do with gitsubmodules.

You can see an example of how I use it here:

In this case this is just linking with the JUCE develop branch, but in my code I have multiple repos with custom modules linked in that way, including versioning requirements, specific commits, etc.

The advantage is that this way of doing it is ā€˜viralā€™, so the knowledge of required versions will get to dependencies of dependencies, etc, and the package manager will be able to resolve a more complicated dependency graph and fetch the correct thing from git.

5

I havenā€™t tried it yet, but iā€™d imagine that when developing own modules based on the JUCE module structure it would still be possible to pre-build and package them- theyā€™d be just static binaries after all, right? Since youā€™re in control in this scenario, you can avoid relying on some compiler flags (like Juce_PluginName or sth similar). In my mind it should be possible to create one big static library from the whole JUCE codebase including all modules, which is then linked against the custom modules. But this is actually a different topic :slight_smile:

Thank you for the suggestion for CPM.cmake, I only now understood how it is intended to be used, it looks very promising. Am i reading this correctly in that it will clone the repository in question into a local folder? E.g. would i be able to check out a branch in the repository downloaded by CPM and make some modifications right away?

Do you have any example how you would have more complicated versioning requirements, e.g. not only relying on a tag but rather CPMAddPackage("gh:catchorg/Catch2@<2.5.0")?

How does CPM handle nested dependencies?
e.g. imagine this situation

product/               -> requires JUCE > 6.0.0
  ext/
    JUCE/
    MyModule1          -> requires JUCE > 6.0.1
    MyModule2          -> requires JUCE > 6.1.0

If i were to move to the CPM approach, iā€™d create git repositories for MyModule1 and MyModule2, and
in their CMakeLists.txt iā€™d add something like the following, as in your example:

CPMAddPackage(NAME JUCE GITHUB_REPOSITORY juce-framework/JUCE GIT_TAG origin/6.0.1)

Is CPM clever enough to notice that both MyModule1 and MyModule2 need JUCE, but with different versions?

Module format: versions of dependencies?
6

Unfortunately no. You can of course do that with your own code if itā€™s not based on JUCE, but once you include any JUCE module (even juce_core) you will have to recompile the module (and all dependency modules) along with your specific app flags.

I agree that itā€™s not so great, and as someone who manages many JUCE based projects it would be amazing if that changes and a static library would be possible, but as of now and the near future I would imagine it wonā€™t be the case.

Yes. The great quality is letā€™s say you have many interconnected modules but want to be maintaining (change) one, you can use:

cmake -G Xcode -B build -D CPM_Toolbox_SOURCE =path/to/toolbox

And then all nested repositories would know to look at your local folder instead of git. Thatā€™s especially important when youā€™re maintaining a dependency of a dependency, which isnā€™t something git submodules know how to handle.

CPM does handle versioning requirements in each dependency, thereā€™s a VERSION argument when calling it - check out the readme on the github page for example.

Yes, butā€¦ it will only really work in a ā€œforwardā€ way. So if you have a module that requires JUCE 5 and a module that requires JUCE 6, you canā€™t have each module build with itā€™s own version.

In that case the package manager will ā€œresolveā€ the dependency requirements and will only clone the very latest version needed, so your modules that required the old versions might not compile if the API changed between those versions.

1 Like
7

Seems like a very nice way of handling non-binary dependencies, iā€™m definitely going to play around with this a bit! Thank you very much for your input!

And stillā€¦ if JUCE would be able to handle module dependencies via the module headers, all this external tooling would be kind of obsoleteā€¦ @reuk do you think that this would be a possibility, is there any interest of integrating such a system into JUCE?