JUCE Modules Compilation Speed - What's the trick?

1

Hey – I’m wondering if anyone has insight into how / why the JUCE modules compile so quickly.

I’ve got a fairly large library with some design flaws in it which stop it from being statically built – at the same time I notice an entire JUCE module compiles in about 10 seconds and has other things stopping static building. The difference is an entire JUCE module compiles in like 15-20 second, but for my library, almost every file takes 5-10 second making for a slow build.

Is there really that big of a speed benefit to moving into a monolithic h & cpp file like in the modules? – I’ve made heavy use of forward declarations and things are pretty isolated in terms of their header spaces, but there’s still a ton of includes in the .cpp files, I thought this would actually be favorable over the single .h & .cpp but it seems I’m wrong.

What’s the trick here – is the entire JUCE module really compiling in 10 seconds because of it’s include layout? Is there a term for this design pattern?

J

2

The short answer is that yes, Unity Builds can massively speed up the compilation times of your library!

This is a pretty good article on Unity Builds: The Magic of Unity Builds · OJ Reeves

2 Likes
3

Unity Build! Awesome thanks for the term for that @ImJimmi

4

FWIW, if you ever need to Google stuff about unity builds… make sure to add “C++” to the front, otherwise you’ll just get results about the Unity game engine :stuck_out_tongue:

3 Likes
5

Also, if you’re using CMake, there’s a buit-in way of enabling Unity builds on your projects that doesn’t require maintaining your own “combined” translation units:

https://cmake.org/cmake/help/latest/variable/CMAKE_UNITY_BUILD.html

This might require a bit of work to get set up in a project that uses JUCE - the JUCE module sources really shouldn’t be combined into even larger TUs, so you might need to set SKIP_UNITY_BUILD_INCLUSION on all of the JUCE source files, or disable the UNITY_BUILD property any target that links directly to a JUCE module.

This technique works best for ‘normal’ static/dynamic/executable target types, so it might not be suitable for this particular use-case. In any case, it’s a useful feature to know about!

7 Likes
6

Omg! This looks amazing – CMake keeps on surprising me, I’m going to give it a shot, maybe there is a way to set it on a specific group of sources – will report back

7

Here’s another interesting approach – in any case I’m gonna experiment with this more.

Thanks everyone!

8

ffs, really gonna have to learn cmake at some point :frowning:

1 Like
9

It’s a pain in the ass and your Xcode projects will look like absolute garbage and you’ll feel like you’ve entered some dystopian hell trying to read their docs – but – if you manage multiple plugins or any sort of build processes like scaffolding generators, the pros out weigh the cons.

Also I just compiled from VS Code for the first time today which was pretty cool!

1 Like
10

Also just for reference to this thread – while I had errors In my compilations with the unity build approach – it was clear when I get it working it’s going to slice my compile times down by a very large amount.

The script I posted above was super helpful, it basically generates a unity build file for you, but it seems not robust of dealing with namespace collision

11

Is that something I can get to work in Xcode for my projects too? :smiley:

12

Turning 99% of my code into JUCE modules has made a dramatic change (for the better) in compile times, and also really helped creating chunks of reusable code that each project can just pull for it’s purposes.

It’s important to note that JUCE-style modules don’t require CMake and you’ll get the same speedup in Projucer builds (even though CMake is way more flexible, etc).

4 Likes
13

In addition to unity builds, we’ve also experimented with precompiled-headers to improve compile times of non-JUCE code:

target_precompile_headers (plugin PRIVATE <juce_core/juce_core.h> 
                                  PRIVATE <juce_dsp/juce_dsp.h> 
                                  ...)

This creates a binary intermediate representation of the headers, so the compiler doesn’t need to parse the header every time a file includes them. You don’t need to touch any sources for this to work, just add the cmake-command and during compilation the PCH is accessed.

One downside of this approach is that every time one of the headers specified in the command changes, all sources that include one of the PCH-headers will need to be recompiled. In case of the juce-sources this probably isn’t a big deal, but when you’re dealing with frequently-changing headers this is something you need to pay attention to.

Also, as with the unity builds, be sure to exclude the juce-sources from accessing the PCH:

file(GLOB_RECURSE JUCE_SOURCES CONFIGURE_DEPENDS ${CMAKE_SOURCE_DIR}/ext/JUCE/*.cpp  ${CMAKE_SOURCE_DIR}/ext/JUCE/*.mm ${CMAKE_SOURCE_DIR}/ext/JUCE/*.r)
set_source_files_properties(${JUCE_SOURCES} PROPERTIES SKIP_PRECOMPILE_HEADERS TRUE SKIP_UNITY_BUILD_INCLUSION TRUE)

You can also create a header-file that simply includes all other headers that you want to be in your PCH, and run the cmake-command on that one:

# pch.h
#include <juce_core/juce_core.h>
#include <juce_dsp/juce_dsp.h>

# CMakeLists.txt
target_precompile_headers (plugin PRIVATE pch.h)

With both approaches combined, we’ve managed to reduce our compile times by about 75%, without touching any source code:

target_precompile_headers(plugin PRIVATE pch.h)
set_target_properties(plugin PROPERTIES UNITY_BUILD ON)
file(GLOB_RECURSE JUCE_SOURCES CONFIGURE_DEPENDS ${CMAKE_SOURCE_DIR}/ext/JUCE/*.cpp  ${CMAKE_SOURCE_DIR}/ext/JUCE/*.mm ${CMAKE_SOURCE_DIR}/ext/JUCE/*.r)
set_source_files_properties(${JUCE_SOURCES} PROPERTIES SKIP_PRECOMPILE_HEADERS TRUE SKIP_UNITY_BUILD_INCLUSION TRUE)

One last thing to mention is that it seems that, at least in our setup, AppleClang that ships with XCode 12 crashes when using PCH, the issue seems to be resolved with XCode 13, but even only with unity builds we’ve reduced compile times by roughly 60%.

2 Likes
14

Is there a tutorial how to do this properly? I’d be also interested in one without using Projucer. Having to create source files in there really slows down my workflow.

15

I probably need to film a tutorial about it!
Generally speaking a module is a .h file including other .h files, and a .cpp file including other .cpp files, with a custom metadata area that signals the projucer or CMake what are the module dependencies.

I have some examples here:

And you can also look at simple JUCE modules like juce_osc.

Once you have a module you can just add it in the modules list in the projucer, or link against it in CMake.

P.S. Despite JUCE’s own modules using ‘implicit’ includes (not including anything in the actual source files or header files) doing it that way causes problems with IDEs, so I’d recommend to avoid it and just include the dependencies of each source files like you would in non-module code.

2 Likes
16

Thanks!
I still don’t quite get how this speeds up compilation time. I’m using Xcode but googling for “Xcode Unity Builds” doesn’t really get me anywhere.
Would be great if you could link an article or tell me more about it if you have the time :slight_smile:

17

@ImJimmi linked an article above. I think mostly it means way less I/O during the compilation.

Pretty much like why a buffer of 1024 samples runs faster than 2 buffers of 512 samples, with unity builds the compiler just has more reusable cache for each operation it does, VS tearing up the cache and creating a new translation unit for each few lines of source code.

1 Like
18

If you can group all highly coupled bits of code into the same compilation units you can also get almost all of the benefits of Link Time Optimisation (LTO) without having to do the usually very slow LTO step afterwards.

2 Likes
19

There’s also a great talk on the subject here: The Hitchhiker’s Guide to Faster Builds

1 Like
20

Thank you everyone for sharing your tips!