[FR] Improve the performance of building juceaide by forwarding compiler launcher CMake args

[ImJimmi]

sccache can be used to greatly improve build times by caching object files. In CMake it can be enabled simply by adding:

find_program(SCCACHE sccache)

if (SCCACHE)
    set(CMAKE_C_COMPILER_LAUNCHER ${SCCACHE} CACHE INTERNAL "")
    set(CMAKE_CXX_COMPILER_LAUNCHER ${SCCACHE} CACHE INTERNAL "")
endif()

If the following two lines are also added to this execute_process() call, sccache can also be used when building juceaide which can massively reduce the CMake configure time.

"-DCMAKE_C_COMPILER_LAUNCHER=${CMAKE_C_COMPILER_LAUNCHER}"
"-DCMAKE_CXX_COMPILER_LAUNCHER=${CMAKE_CXX_COMPILER_LAUNCHER}"

Here’s an example CI workflow that demonstrates a 25% reduction to the configure time for both macOS (clang) and Windows (GNU):

Add example CI workflow with and without suggested changes

And here’s a diff with the suggested changes: sccache-juceaide/juce.diff at main · ImJimmi/sccache-juceaide · GitHub

[sudara]

Love seeing this kind of stuff, thanks for sharing!

Pamplejuce runs Windows JUCE configure in ~45 seconds — which is much faster than the 4 to 5 minute times you are seeing, so I looked into why…

Looking at your GitHub Action run, it’s using MinGW, no longer really supported by JUCE. I think this is probably accidental, see CMake+Ninja detects GCC as the default compiler on Windows · Issue #1621 · actions/runner-images · GitHub

If you switch to clang, configure will drop to under a minute and with your patch, Windows configure will drop to 30s!

MSVC is fastest (Can be 30-60s out of the box, for some reason your patch is failing to build there). Annoying, but to use it, you’ll need to tell GA to setup the toolchain to use the developer command prompt.

Bonus: This thread made me realize I don’t need to set CMAKE_BUILD_PARALLEL_LEVEL anymore — Ninja has sane defaults and sets it to the number of processors available.

[ImJimmi]

Thanks for taking a look!

Yeah I noticed that GCC was painfully slow - and then tried to use it to compile a VST3 project and it simply wasn’t having it. Spent a long time trying to set up MSVC with Ninja, but after looking at your fork switched to using this GHA to enable Clang on Windows and it works really well!

Something that stood out to me about your fork though is that in the “Post Run sccache-cache” step, it seems to imply that the caching failed:

Compile requests                      7
Compile requests executed             0
Non-cacheable calls                   7

Non-cacheable reasons:
Can't handle UnknownFlag arguments with -Xclang       7

So I’ve a feeling the 30s difference you saw between the first and second run was just a fluke! Possibly there’s some other local caching of the CMake config happening somewhere?

[ImJimmi]

Updated my repo to use Clang on Windows - seeming similar times to @sudara, but also getting the errors about unknown flag arguments with xclang:

Fix yaml errors

I’ve a feeling the juceaide build is silently failing.

[officialnsa]

Hey @ImJimmi, did you ever get to the bottom of those -Xclang errors with sccache? I’m facing a similar issue on my end and haven’t been able to find a solution. I’ve seen mention of similar problems in the sccache GH issues but the issues were all marked as resolved.

[ImJimmi]

I’m afraid not, no!

We have revisited sccache recently but this time are just sticking with MSVC on Windows and using the Ninja Multi-Config generator - I believe all that’s needed to get it working is to enable the Visual Studio dev environment (vcvars.bat).

We’ve basically written our own implementation of Setup MSVC Developer Command Prompt · Actions · GitHub Marketplace · GitHub

[oli1]

Thanks for requesting this. It makes quite the difference on my machine.

This is now on develop:

CMake: Passthrough CMAKE_XXX_COMPILER_LAUNCHER when configuring juceaide

[sudara]

After this change, I see issues with sccache on windows MSVC at configure time.

[3/12] Building CXX object
  extras\Build\juceaide\CMakeFiles\juceaide.dir\__\__\__\modules\juce_graphics\juce_graphics_Harfbuzz.cpp.obj


  FAILED:
  extras/Build/juceaide/CMakeFiles/juceaide.dir/__/__/__/modules/juce_graphics/juce_graphics_Harfbuzz.cpp.obj


  sccache
  C:\PROGRA~1\MICROS~2\2022\ENTERP~1\VC\Tools\MSVC\1442~1.344\bin\Hostx64\x64\cl.exe
  /nologo /TP -DDEBUG=1 -DJUCE_DISABLE_JUCE_VERSION_PRINTING=1
  -DJUCE_GLOBAL_MODULE_SETTINGS_INCLUDED=1
  -DJUCE_MODULE_AVAILABLE_juce_build_tools=1
  -DJUCE_MODULE_AVAILABLE_juce_core=1
  -DJUCE_MODULE_AVAILABLE_juce_data_structures=1
  -DJUCE_MODULE_AVAILABLE_juce_events=1
  -DJUCE_MODULE_AVAILABLE_juce_graphics=1
  -DJUCE_MODULE_AVAILABLE_juce_gui_basics=1
  -DJUCE_SILENCE_XCODE_15_LINKER_WARNING=1 -DJUCE_STANDALONE_APPLICATION=1
  -DJUCE_USE_CURL=0 -D_CONSOLE=1 -D_DEBUG=1
  -ID:\a\pamplejuce\pamplejuce\Builds\JUCE\tools\extras\Build\juceaide\juceaide_artefacts\JuceLibraryCode
  -ID:\a\pamplejuce\pamplejuce\JUCE\extras\Build
  -ID:\a\pamplejuce\pamplejuce\JUCE\modules /DWIN32 /D_WINDOWS /EHsc
  -std:c++17 -MTd /bigobj /Od /Zi /MP /EHsc /W4 /showIncludes
  /Foextras\Build\juceaide\CMakeFiles\juceaide.dir\__\__\__\modules\juce_graphics\juce_graphics_Harfbuzz.cpp.obj
  /Fdextras\Build\juceaide\CMakeFiles\juceaide.dir\ /FS -c
  D:\a\pamplejuce\pamplejuce\JUCE\modules\juce_graphics\juce_graphics_Harfbuzz.cpp



  D:\a\pamplejuce\pamplejuce\JUCE\modules\juce_graphics\juce_graphics_Harfbuzz.cpp:
  fatal error C1041: cannot open program database
  'D:\a\pamplejuce\pamplejuce\Builds\JUCE\tools\extras\Build\juceaide\CMakeFiles\juceaide.dir\vc140.pdb';
  if multiple CL.EXE write to the same .PDB file, please use /FS

Interestingly, /FS is specified, but perhaps the order of flags matters?

[sudara]

According to sccache, the following need to be set on CMake > 3.25:

set(CMAKE_MSVC_DEBUG_INFORMATION_FORMAT Embedded)
cmake_policy(SET CMP0141 NEW)

or alternatively, one can manually do the following:

    if (MSVC)
        if (CMAKE_BUILD_TYPE STREQUAL "Debug")
            string(REPLACE "/Zi" "/Z7" CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG}")
        elseif (CMAKE_BUILD_TYPE STREQUAL "Release")
            string(REPLACE "/Zi" "/Z7" CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE}")
        endif ()
    endif ()

Unfortunately, I can’t apply this from the juceaide invoked build from my main CMakeLists.txt, so the /Zi flag is still set.

I will experiment with modifying juceaide directly to confirm it resolves the issue.

In general, it would be nice to be able to be able to generically pass through some cmake options to the invoked juiceaide build, similar to PASSTHROUGH_ARGS but arbitrary, giving us a bit more flexibility/control over that process.

[andrewj]

Yeah I’ve just been struggling with this and worked around it for now by reverting from Ninja to MSVC. Would be great if we could get some speedier builds back again!

BTW thanks for your blog and pamplejuce examples Sudara, I have been leaning heavily on those!

[sudara]

@andrewj Ah, awesome, good point! I should have mentioned I worked around the issue in another project by removing Ninja as well… Figured I’d try and solve the issue in pamplejuce now…

Unfortunately, modifying juceaide’s CMakeLists.txt to include this didn’t work:

set(CMAKE_MSVC_DEBUG_INFORMATION_FORMAT Embedded)
cmake_policy(SET CMP0141 NEW)

[ImJimmi]

That’s concerning- we’ve just spent about 3 months revamping our CI including moving to Ninja on Windows! We were hoping to move to JUCE 8 as the next step, but looks like that might not be possible now

[sudara]

@ImJimmi — Well, someone should probably double check my work, it’s entirely possible I’ve missed something. I didn’t do extensive flag debugging (yet?).

Also of note: the reason /Z7 should work in the first place is that it embeds debug symbols in the obj files themselves, not a pdb. That might not be a desirable consequence, even if the fix works.

[sudara]

Ok, mostly figured it out after (too many :/) hours digging in a few more layers.

This issue on the sccache repo helped clarify the problem.

It turns out that it’s not juceaide’s CMakeLists.txt setting the compile options, but instead JUCEHelperTargets.cmake.

It sets /Zi, which instructs the creation of a separate pdb file, which makes it cache-unfriendly when building in parallel, as multiple build processes are trying to write to it.

Removing /Zi got me building nice and cached again!

Note, this changes the behavior to not producing a .pdb. I’m not sure if this matters for plugins, since we are middleware and mostly not getting crash dumps and therefore have no ability to hydrate with debug symbols… However, it does put whatever debug symbols exist into the obj files, and therefore the binary. So I’m not quite sure yet if that means it’s going to bloat the juce modules in size with debug symbols, I have to do more testing.

Side question I ended up having: Would it be faster if juceaide was built as Release when building the main project in Release? Right now its always built in Debug. Caveat: I have 0 idea about all the mechanics involved, I’m guessing it’s not considered relevant as the actual modules are built in Release?..

[sudara]

Just to contribute the last piece of the puzzle:

/Zi is being set only as an option in Debug, as part of juce_recommended_config_flags — so one could always just not link against that target to get around it — except in this case, we don’t have control over what juceaide links to.

One way of framing: caching my Release build in CI/Windows is choking on Debug flags.

(That made me check into juceaide build times out of curiosity. And yes, building juceaide in Release takes 2x as long, as expected.)

My earlier mention of obj files and binary size is probably not relevant? The juce module files juceaide builds in Debug are used to prep things like binary data and AU preprocessor stuff — the actual juce headers built into my plugin binary are built in Release, where that flag doesn’t exist anyway.

On my own projects, I’ll just remove the /Zi flag from juce_recommended_config_flags on a fork.

I’m not quite sure what I’ll do about Pamplejuce. I don’t want to run it off a JUCE fork. And sccache doesn’t seem happy on plain MSVC without Ninja…

[eyalamir]

You don’t need to fork JUCE to create your own version of juce_recommended_config_flags. You can just create your own interface target for configs and copy-paste whatever settings you want or don’t want from the JUCE one, it’s just a few lines.

[sudara]

Yes, that would be great if I needed to set flags for my own build. Unfortunately, the issue is juceaide’s own cmake build, which links against juce_recommended_config_flags, setting /Zi in Debug (unhappy in in the msvc/ninja/ccache context).

Although hmm, maybe I could actually redefine the options set by the juce_recommended_config_flags target itself…

EDIT: no, this doesn’t work because juceaide spins up its own configure/build process.

[sudara]

I think the only solution for sccache on MSVC is to petition the JUCE team to:

1. remove the /Zi option from juce_recommended_config_flags in Debug
   or
2. allow us to set general passthrough cmake options to juceaide (can then override the /Zi flag set by juce_recommended_config_flags with /Z7?)

[oli1]

I’m looking into it. Thanks for the investigative work!

[oli1]

Juceaide is now aware of policy CMP0141 and CMAKE_MSVC_DEBUG_INFORMATION_FORMAT.

set(CMAKE_MSVC_DEBUG_INFORMATION_FORMAT Embedded)
cmake_policy(SET CMP0141 NEW)

The work is on develop

CMake: Make juce_recommended_config_flags CMP0141 aware