Just as a reference BTW, the VST3 SDK uses unity builds:
Yet the “unity” .cpp file has no includes to ‘global’ .h files, and instead each .cpp files includes files the plain old way:
They also moved a bunch of all the common preprocessor into a common header:
Which is always in the include chain (say, through fobject.h) of every header in the library.
VSCode and other IDEs have 0 problems parsing the code in the VST3 SDK because of it.
Please reread the discussion. I’m not suggesting to switch away from unity builds. In fact I like unity builds and they are generally the fastest way to build.
I’m just asking to change the includes in a simple way, similar to how it’s done in the VST3 SDK, or in my own JUCE modules (that also use unity builds).
juce_osc has 8 headers and 8 source files with no real complexity, I don’t think a test with that module is fair. juce_core or juce_gui_basics would be much better tests.
To be clear we think this would be good because
To me it seems like the only benefits would be for JUCE devs or developers making changes to JUCE forks, on which the JUCE devs don’t encounter as many issues as you’re describing. I use Xcode daily - yes sometimes it chokes but it does with any C++ code base, a quick clean or quit and restart normally does the trick for me. In the team others use VS and CLion daily too.
To be clear I’m very aware of how most C++ projects are structured and honestly I’ve always preferred the JUCE modules 
To the point that I’ve even made projects where the source of my application or plugin has sometimes been implemented as a JUCE module. that way I just add a file to disk resave and there’s nothing else to change, no file to add to CMake or the Projucer, and I don’t need to tell it how to organise the files in the IDE. I realise however it’s not for everybody.
I believe these are the only two advantages, but they’re substantial. Build times would probably be identical or slightly faster and IDE parsing time and accuracy would be dramatically better and faster.
It would be a huge improvement to anyone using JUCE, because auto-complete would be dramatically faster, and the IDE would spend way less resources parsing the build system recursively.
Also I (and probably other people) spend quite a lot of time browsing the JUCE code base and doing ‘go to definition’ on things which currently kinda-works in CLion but many times will break.
Here’s what I suggest:
All I need from the JUCE team is an a promise to review that once it’s done. That would give me an OK that this kind of change would work, and then I’m happy to send a PR for other more complicated modules.
That kind of PR would be simple, isolated, non-breaking and can work as-is without changing anything in other modules.
juce_core would be a much bigger PR as it would require a bunch of new .h files created, so I’m only willing to do that if the JUCE team agrees on the format for such PRs, and will agree to review it.I’m asking for that part to remain exactly the same as it is today with no changes to the build system. Unity builds should not go away. Projucer builds should still work exactly as-is, you will still be able to just add juce_osc to your project, etc.
We maintain about 40-50 custom modules currently.
All are written in JUCE-module style with a single .cpp and a module header with metadata. All work in the current CMake/Projucer system and all have added explicit includes which works great, including in modules that wrap external libraries and don’t depend on JUCE.
So I went about creating an extremely simple example of a project built using the unity build style JUCE employees and I’m finding that as long as I follow a couple of key rules in CMake it works fine in Xcode, VSCode, and CLion on my Mac, I haven’t tried anything on Windows just yet.
Is the problem only that it’s too slow in IDEs? in which case would splitting the modules into smaller modules potentially have a better pay off than adding all these header includes?
The problem is once the project gets complicated with multiple translation units the IDE either can’t parse it completely (show red squiggly lines in VS Code) or just takes a ton of memory/time to do so (slow auto complete in xcode, no syntax highlighting in CLion, etc).
You can see a ton of screenshots in the threads I linked to, this has been reported many many times in the forum on all platforms and IDEs.
The simplest way to test this is to run any JUCE example and look at JUCE source files in VS Code.
@anthony-nicholls I’ve created a simplified project faking the module system and showing the problem in non-JUCE code.
There are two folders there, one with unity builds the way JUCE is structured, and one with my suggestion but also with unity builds.
Since this project is super trivial, you will see CLion for example figure it out pretty quickly.
VS Code however will struggle. I’ve also noticed that it will sometimes attempt to re-parse it with its own maybe folder searching, but it will fail again later since it has two conflicting ways to answer the question if the code will compile or not (with and without the includes).
All IDEs are super fast in parsing the “with includes” version.
JUCE-Fake-Modules.zip (17.7 KB)
Edit: You may also need to add
set(CMAKE_EXPORT_COMPILE_COMMANDS TRUE)
To the CMake file in both versions if testing in VS Code, I forgot to add that.
To expand on what I said
OK but what I want to check is that we actually understand the root cause, “gets complicated with multiple translation units” isn’t specific enough in understanding the problem.
Yes but we can also see there are multiple causes of this, VS has a fix on the way, Xcode demonstrates problems like this in lots of C++ projects, and not everybody is experiencing the same issues so evidently there must be more to it even if adding the includes appears to fix the issue for you.
I’ll do this next but for example in my simple experiment I was able to find that
HEADER_FILE_ONLY property seemed to allow browsing the source againHEADER_FILE_ONLY propertyLooking at the JUCE source code the HEADER_FILE_ONLY property is only set when JUCE_ENABLE_MODULE_SOURCE_GROUPS is set.
That being said I’ve just created a JUCE project and all seems to be working for me, I do see some red squiggly lines in VSCode but I can jump to declarations and definitions without any problems, and it’s super fast in both VSCode and CLion.
Maybe if you could share a project you or anyone else is seeing issues with?
This is the root of the problem.
“Dumb” text-parsing IDEs don’t know the build system.
They look at 'MyCode.cpp", and try to parse all the types through it.
All IDEs have all kinds of hacks to fix that. They will try to first parse the headers from the .cpp file, then they’ll look at other files in the folder, and eventually in the build system if they can (non-VSCode).
Their success in doing that is partial, and indeed the problem gets worse the more complicated the build system is.
Yes, if you have one very obvious target, the IDEs can probably find all the headers in it. This gets difficult with JUCE (that has in a plugin a shared target + many interface targets + the actual VST3 target), so it becomes harder for the IDE to find what’s a ‘source file’ through the context.
Everyone who have tested my suggestion back then for juce_osc back in the day reported it works. This isn’t a “Works for me” or “my code style” thing but a consensus “Good C++” practice suggested everywhere in the community.
For example, this blog by Incredibuild:
It’s also recommended by the google guidelines:
https://google.github.io/styleguide/cppguide.html#Include_What_You_Use
And many other places around the web.
Those are bugs. Correct code that compiles correctly shouldn’t show red squiggly lines. The fact that I (or the IDE) can work around does not mean they’re not bugs.
Those squiggly lines usually mean the IDE model for the code is not correct so analysis and refactor tools will not be correct either.
This is VS 2022 17.7.6 with a Projucer-generated project (7.0.8) on Win 10:
This is not “some red squiggly lines”. It’s painful. All of JUCE cpps show like this.
I think this is discussed above my understanding is it’s a VS bug with a fix on the way?
I’m not sure they actually consider it a bug. The supposed fix has been “on the way” for more than a year. It seems to me that whatever they changed back then, they were aware of the issues it may cause in cases like these, and went with it anyway.
The only way I’ve ever (6+ years) managed to get no squiggles in Visual Studio inside of JUCE cpp files is with ReSharper (which costs money + spins up a ton of extra resources and slows the system down). This is in VS 2015, 2017, 2019 and now 2022.
Unsurprisingly, with my own JUCE-style modules that have explicit includes, VS without Resharper can parse them just fine with no squiggles.
To be clear I am looking into this but I assure you that for complex modules like juce_gui_basics it’s not as simple as it’s made out, obviously a module like juce_osc is very simple. I have also done some tests and found it’s not penalty free, using a very naive and albeit inefficient method I measured a 6% increase in build times for juce_gui_basics on my machine, and that’s just because of all the pragma once look ups! I haven’t done extensive testing yet but that kind of increase does matter to a lot of customers. I’m also finding that simply adding the includes does NOT always fix it for all IDEs. At the moment I suspect because of sheer size of the modules. The first step I would like to take is to break up some of our more complex modules in order to at least make this work easier, this also has the benefit of improving build times and potential reducing the build time of buildaide quite significantly.
@anthony-nicholls thank you for looking into this!
If you don’t mind I’d love to review how you’ve done it and see what the problem is.
Even in a huge module like juce_gui_basics, most of the #include could be boiled to the exact same one like juce_component.h.
That module only has 414 source files, probably 90% of these includes would be the same one.
Just like any C++ source code, you don’t want to have extra includes in there if they just get discarded. You can organize and condense them in a way that makes sense.
I think the devil in the details and exactly how the includes are
structured. Even though it’s a (way too) large module, it should mostly boil down to a lot of repetitive includes of the same header that should be very cheap and cached easily.
Fantastic! I think if you manage to move juce_Component (along with Layout, LookAndFeel, Accessibility, Mouse, etc) to a separate module from all the Widgets, Buttons, etc, that would be a good start.
I warn you it’s a horrible hack that is massively inefficient so I would expect it to be an absolute worse case scenario and doing it “properly” should be significantly more efficient, but it should essentially just add a lot of redundant includes that should be immediately ignored compared to the current setup.
The basic rules are
What this means is
This effectively turns every source file into the module source file that includes it when they are compiled or parsed individually.
Looking back I should have seen what difference it makes JUST adding the pragma once even if they aren’t ever doing anything.
If you apply this technique to the examples you shared it appears to “fix” the issue. I can tell you I could still reproduce issues in VSCode and especially CLion with JUCE after trying this.
Another very simple test I done was add some basic includes to a single file for example adding “juce_Component.h” to the top of “juce_Component.cpp” and it appeared to make no difference to the ability to navigate to the declaration of Component methods in CLion for example. I haven’t tried again but speaking to other devs I might’ve had a better experience if I adjusted some settings to allow it to use more RAM in CLion. interestingly CLion had no problems with juce_osc without doing anything.
Thank you!
But wait…
Does that mean you’re adding #pragma once to .cpp files? Don’t do that!
If you look at the example (with includes) I’ve added, the rules should be:
Remove #include <module_name.h> from the top of module_name.cpp.
That part is crucial, because it will mean the module will stop compiling, forcing you to add includes correctly.
Move all the common preprocessor, external includes into one config.h.
Start adding includes in the simplest way possible (for example, add juce_Component.h to juce_Component.cpp and juce_Component.h to juce_Button.h while adding #pragma once to all .h files (But not to a .cpp file).
Following these 3 rules (or looking at my with_includes example that does that. Should get you on the right path to something that will compile fast and be trivial to parse.
Right, that won’t work, you need juce_Component.h to start including everything it needs, like <juce_graphics/juce_graphics.h>
Wow…
