Native, built-in CMake Support in JUCE

Another reason this can occur is when using CPM to manage a JUCE module. For example, I diagnosed why this error occurred for me with the FFMeters JUCE module by doing the following in the JUCEModuleSupport.cmake file:

message("FFMeters PATH is: ${module_path}/${module_header_name}")
if(NOT EXISTS "${module_path}/${module_header_name}")
    message(FATAL_ERROR "Could not locate module header for module '${module_path}'")
endif()

Which resulted in:

FFMeters PATH is: C:/Users/zendev/repos/pinpoint-apps/cmake-build-debug/_deps/ff_meters-src/ff_meters-src.hpp

Note that the header I want it to find is ff_meters.h, and that the .hpp is irrelevant since the JUCE CMake tries for .h first and then tries .hpp if .h is not found.

So the problem is the interaction between where CPM puts the dependency (in a folder that appends ‘-src’ to the end of the Git repo name, and what JUCE expects.

@reuk I think an appropriate solution here might be to follow the style of the .h/.hpp logic and also try looking for a filename that discards the ‘-src’ suffix if it is detected. There is a risk of the CMake getting a bit messy here but it might be worth it to ensure CPM support for JUCE modules OOB.

For the record I am hoping to migrate over to vcpkg from CPM but I would rather do it all in one shot and CPM is what we’ve been using for some time now.

I’ll post the relevant changes next, in case it can be added to JUCE upstream.

Here is the relevant fix, which makes sense to place before the check which tries .hpp if .h module header is not found. I am pretty sure there are some edge cases to worry about here I’m not considering but this did fix my issue. A more robust solution would check that “-src” is at the end of the string, rather than removing it wherever it appears, but I don’t have the time to get into regex…

if(NOT EXISTS "${module_path}/${module_header_name}")
    string(REPLACE "-src" "" module_header_name ${module_header_name})
endif()

@eyalamir I’d be surprised if you’ve not run into this before! I suppose on the CPM side, you might be able to customize the name of the folder it clones the repo to. But fixing it there would remove the symmetry with the -build and -subbuild folders.

Update: It looks like the problem is more severe, the Unity build of the module sort of falls apart when only applying this band-aid. It might be the case after all that fixing things on the CPM side gets it working with fewest needed changes. On the JUCE side, I guess the important thing to make clear is that JUCE modules make an implicit assumption that the module name and parent folder name are identical.

Yes, as you’ve discovered, JUCE assumes that the module folder has the same name as the module, and CPM by default will create other names. There is a CPM setting you can set - if you set the variable CPM_USE_NAMED_CACHE_DIRECTORIES to ON before including CPM, you avoid this issue.

In my opinion, the best solution is for someone writing a JUCE module to not put the module header in the root of the repository, but in a named subdirectory, so that you always have full control and can ensure that the module folder has the same name as the module.

For example, for a module named foo, I might have:

foo/ <- repo root
  CMakeLists.txt
  foo/ <- module folder
    foo.h
2 Likes

Yeah that sounds about right, I’d seen this variable used internally for other purposes but I can see why it makes sense to use it in this scenario as well. Still, based on my initial test of vcpkg it seemed that this entire category of problem disappears completely, so feeling pretty motivated now to migrate over.

Are you using vcpkg to just fetch the source code of these third party juce modules? I know vcpkg supports fetching prebuilt binaries, but of course juce modules don’t support being built as static libraries.

Yeah - this gets back to our earlier convo in the TAP Discord (build-systems channel). If I did something like this, I would try to implement a PoC refactoring the entire Unity Build system to a flexible static library system, allowing for deferring combining of per-module static libs until as late in the build process as possible in order to deduplicate JUCE modules and 3rd-party modules no matter what way they are used across lower level libraries and higher level apps and executables.

I wouldn’t have had the confidence to do this had I not seen how it is possible in other complex CMake environments like Tensorflow Lite… if done right it could probably stand alongside the Unity Build system rather than replace it, so that it would maintain backwards compatibility while providing a good path forward.

In other words, you could write a generic adapter that would work for any legacy JUCE module so long as it adhered to certain assumptions correctly - which is the same problem we tend to have anyway with that legacy system.

The main issue preventing you from building a JUCE module as a static library is the usage of so many configuration preprocessor macros. The build system alone can’t fix this, the juce C++ code itself would need to be changed to not use these macros.

Unfortunately this doesn’t seem to work in this situation, I still get the same error…
Here’s what I have in my top-level CMakeLists.txt in the relevant section:

set(CPM_USE_NAMED_CACHE_DIRECTORIES ON)
include(CPM)
find_package(JUCE)
find_package(ff_meters)

Yeah, there would need to be a bit of back-and-forth but hopefully the opportunity will arrive in the Fall if a good solution architecture is evident that takes all these considerations into account. The fact of the matter is that a really meticulous decoupling of front end and back end code within the JUCE modules combined with something like this would allow for some pretty powerful scenarios wrt migrating an audio back end to different processors while abstracting away the UI implementation.

If the current audio hardware landscape is any indication, this is going to be a big thing in the years to come.

I personally don’t expect the juce module system to change any time soon.

This was the same point I was trying to make on the TAP Discord - if you’re unhappy with something about JUCE, you can work on a PoC to show a possible way forward, as I’m tempted to do in this case. I think the same was the case for @mcmartin wanting CMake support in the first place, which led to GitHub - McMartin/FRUT: Building JUCE projects using CMake made easy and in turn probably led to native CMake support provided by JUCE themselves (this is why I did not give credit to FRUT at ADC 2021 but probably should have…).

The same was the case back in the day when Vinnie Falco wrote a lot of code that wound up in JUCE itself (try getting vflib to work with modern JUCE and you’ll see just how much the namespaces collide!)

So it is always possible to solve the problems you find in JUCE yourself… or at least enough to offer a PoC. If it’s too much work to do the whole thing then a PoC is a proactive way to getting the changes you want into JUCE without overextending your own time commitment to this type of a project.

Changing the internal usage of configuration macros requires changes to existing juce modules, not just adding more code/tooling. And it would be substantial enough changes to amount to basically an entire rewrite/redesign of many of the core juce modules.

I could spend the time to make an enormous PR for that, but it would never be merged during my lifetime, so I think my time is best spent working with the existing module/unity build system, and accepting that it’s a necessary relic of past design decisions that JUCE is pretty much locked into for the foreseeable future.

1 Like

All the juce modules I’m fetching just have another subfolder in the git repo with the name of the module.

I think you can just ask @daniel or other maintainers of the modules you’re using to change the file structure of the repo and solve that problem.

Usually providers of 3rd party JUCE modules (gin, tracktion engine, etc) do it like that anyway, so module contents is not in the root of the repo, and therefor adding it to JUCE doesn’t require a specific package manager setup.

1 Like

I guess it depends how deep you go with JUCE, but I do think the juce_core module is a good choice for showing a PoC towards a new code organization strategy that permits the kind of flexibility we’re both looking for here in building JUCE as combinable per-module static libs. Another good example of a code organization strategy that works really well in this capacity is Boost itself, and Vinnie’s work back in the day really showed the opportunity space for where this kind of thing could go.

Ah yeah I guess this is also the first time I’m come across this edge case as well then. It’s a non-issue in the end as it made more sense to drastically adapt the code anyway rather than using the module as-is, but it made me think again of this topic of general design strategies for back end (and now also front end) libraries when it comes to using JUCE, that could potentially evolve from the current approach.

I think a good PoC for a back end solution would be showing how juce_core could be refactored to meet the goals outlined above, while a good PoC for a front end solution would adapt the Component system to adhere to a “bottom-up” approach rather than a “top-down” approach where a Component at the bottom of the component tree describes its own size to the parent to perform the layout, and so on and so forth. By combining both of these designs with a modern 3D engine like WebGL/WebGPU you’d have a very good overall architecture I think.

You’re describing rewriting basically all of JUCE. At that point you might as well create your own framework instead of still calling it JUCE.

It might be worth poking around the forum for records of the early days as that certainly wasn’t Vinnie’s attitude when writing vflib, which as said was largely integrated into JUCE. To integrate it to JUCE 7, you need to make modifications like unfinalizing certain low level classes for drawing, GraphicsContext, etc. I am sure the Direct2D work required similar changes. Direct2D Part Deux : 2 Direct 2 Furious

PoCs in this regard however are a lot less than that, kind of like a minimum LoC example to demonstrate a bug in current JUCE master or develop.

We have a need to work with juce_core in a very “tear it apart to get it to work in resource-constrained embedded environments” way as it is, so it’s not much more effort to play around with it.

Demonstrating an IoC paradigm for the Component system also doesn’t have to be that much work… a few subclasses on top of a rewritten juce::Component class yielding a demo that shows the advantages is all that’s needed.

You seem really optimistic about getting these changes into upstream JUCE, which I agree would be great if it happened. So I encourage you to create a PoC PR, go for it! But I cannot afford to spend my time that way.

1 Like

I am sure the Direct2D work required similar changes. Direct2D Part Deux : 2 Direct 2 Furious

Bender