Currently, IDEs have trouble parsing JUCE code due to the lack of explicit includes in modules.
This is detailed in many discussions on the forum, for example here:
This change is non-breaking, will not change the unity builds system, and can be done in a few hours of work.
It would be a major lifestyle improvement for all devs using JUCE, and probably to devs working on JUCE, too.
Hello peepls! I strongly second this!
Anyone who thirds this?
third!
btw it is much easier, no tool required. just remove the juce_foo.h from the juce_foo.cpp and start addingâŠ
A drawback could be that currently it is guaranteed that all config defines are included thanks to the master module header
This should be guaranteed too:
Add some header file like config.h add all the âexternalâ includes to it, as well as #define macros.
Make sure each .h file either includes config.h or another .h file that includes config.h.
Compile to verify - not having those includes in the path will very likely prevent compilation, you can also compile existing projects and unit tests to double-check.
Previously discussed here:
I stand by my assertion that it is not straightforward to add all of the necessary explicit includes. Doing this the naive way (removing includes of .h files from the top-level module .cpp, then adding them to inner .cpp files until the build works again) is not guaranteed to produce correct results, because includes in âearlierâ .cpp files will still be visible in âlaterâ .cpp files.
Moving all internal includes to a âconfig.hâ file that gets included in every .cpp is not guaranteed fix this problem. If the config.h gets forgotten in any .cpp that appears later on in the top-level .cpp, the module will still build because of transitive includes through earlier .cpp files.
It also sounds like completion in VS Code may also currently be broken for JUCE headers, too (can anyone verify?). Therefore, weâd need to ensure that each internal header also included all the necessary dependencies, which again would be difficult due to transitive includes.
I think the only way to guarantee correctness would be to add a non-unity build mode for each module, and to test that mode in CI in combination with other commonly-used preprocessor options. This would take much longer than a couple of hours to put together, and would incur substantial ongoing maintenance costs.
In short, Iâm not convinced that the benefits outweigh the costs. It also seems as though any benefits would mostly apply when working on first-party JUCE modules, so Iâm interested to hear some concrete examples of problems that would be fixed by this change.
Thank you for your suggestion @reuk.
I understand your concerns about the âperfectâ way to do this with a tool that would 100% guarantee such a change would indeed include all needed .h files.
However, since this a change that even if done without 100% proofing of that would dramatically improve the day-to-day life of all JUCE devs, My suggestion is to make that change ânaivelyâ, and just incrementally fix the very rare undetected issues that would happen as a result.
I suggest that since the âworst case scenarioâ here would be a few red squiggles that are left on a handful of .cpp files (and probably reported by the community and fixed fast), while the upside would be that most JUCE files would work just fine as-is.
Currently because of this problem, IDEs (even âhigh endâ ones like CLion) sometime âscrew upâ things like go-to definition. Itâs also a big problem when browsing JUCE code. auto complete and other IDE refactoring tools are randomly breaking because of this, etc.
That is just due to VS Code naively trying to recursively parse headers. It may find juce_Component.h, but it might not be able to find juce_String.h Even though theyâre all included in transitive includes of juce_gui_basics.h.
That is also whatâs causing IDEs like Xcode/CLion/etc to spin up a lot more memory to find how to index functions like Component::getName() and eventually put them in the auto-complete box, as usually these IDEs cache the database of a header by parsing it and then parsing all #includes coming from it recursively.
To simplify, hereâs my suggestion:
Do the change ânaivelyâ (just add the includes, remove #include module.h from module.cpp).
Test builds exactly the way theyâre tested now. If this change would somehow cause a user reporting an error due to a combination of flags, that probably means a test for that set of flags needs to be added to CI anyway.
Make the change incrementally, starting with simple high level modules like `juce_product_unlockingâ, âjuce_oscâ, âjuce_dspâ etc, splitting this into well encapsulated PRs so that over time that change would lead to a great benefit the JUCE community as a whole, many of which have reported the problems caused by this for years.
There were recent reports of Visual Studio and Intellisense being broken when parsing JUCE module code, but this has been fixed and is awaiting release (linked below). Perhaps this fix will also apply to VS Code.
https://developercommunity.visualstudio.com/t/Intellisense-Errors-in-CPP-File-underli/10114557
Those arenât related. I experience these issues in CLion all the time. The problems arenât just VS-code specific. It breaks a lot of IDE systems, including go-to-definition, auto-complete and others.
For example many JUCE .cpp files in CLion look like this (no type highlighting, canât use go-to-definition, etc):
This issue happened in my own modules too:
And solved instantly by adding includes.
Just for fun, here are other plenty of cases where similar problems were reported in all other IDEs:
It goes onâŠ
Please @reuk. No other codebase in the world adds tests for a build system that isnât supported just to add #includes for a few .cpp files.
Adding those is non-breaking, simple to do (can be done per module in an isolated way), can be iterative and will probably be a huge improvement to the lives of everyone here.
Letâs not make this more complicated than it is. Transitive includes is a common mistake that can happen in every C++ codebase out there (and it happened once in a while in my own code) and the fixes are trivial, similar to documentation errors or typos.
Please consider fixing this as this has been reported many, many times by users in all OSâs and all IDEs, and I bet the JUCE team experiences this on a daily basis.
I think it would be nice to have a small concrete example of the actual problem so we know what weâre does actually fix the issue. There could be a lot of work for very little reward if we donât understand the actual issue. Also I assume this is only while browsing JUCE implementation code? If users arenât employing unity builds and they only browse the headers then itâs not an issue right?
Itâs worth pointing out that any work to resolve it will also mean extra time to compile due to all the discarded headers. It would be good to understand the trade off too.
This issue is shown in many, many forum posts (see above).
Easiest way to observe this is just browsing JUCE implementation files in VS Code, but also in CLion, Xcode, etc.
Thatâs not true, itâs the other way around actually.
Since the unity builds system wonât (and shouldnât!) change, all it would mean is that in the same compilation unit (say, juce_core.cpp) there would be a bunch of lookups for, say, juce_string.h.
Since that file would have #pragma once, that call would do nothing. The compiler isnât re-parsing a header that already exists in the same compilation unit.
In the case of an IDE the result would mean dramatically faster auto complete and code cache, since the IDE knows to read include guards, which currently it will ignore.
Iâve tested this suggestion in all my modules that at one point of time converted to explicit includes. Build times, IDE auto complete time, etc, became dramatically faster instantly.
The C++ compiler (and all tools based on it, like clangd) knows how to work with trivially written .cpp files, including how to parse the preprocessor efficiently.
A #Pragma once costs itâs not entirely free, I grant you itâs probably minimal but we could be talking about a lot of them!
#include <juce_component.h>. They donât need to explicitly include <juce_graphics.h> that is a requirement of it.So what the compiler will have to parse is the âhigh levelâ include and it wonât need to even test for all the internal includes that are brought in as a part of this module.
As far as the compiler is concerned, thereâs some std::map<std::string, bool> that tells the compiler to do nothing. You can benchmark this and see that even a million such lookups are probably in the time area of non-measurable difference
All of the above is related to a clean compilation.
In an incremental compilation thatâs done after only some files have been changed, the compile process would be much faster since unless youâve changed something in the #include dirty list, the compiler can decide to do nothing.
Currently, the current system is fast to compile as a whole but the compiler has to do a lot of work parsing the entire module for each change since it has no idea which headers from it can be completely ignored in the compile process.
I suspect that the speed change of incremental compilations in modules would be something the JUCE team (or anyone writing custom JUCE modules) would also see instantly.
Correct me if Iâm wrong but I expect Component will need plenty of other things included too.
More than happy to be proven wrong here, but my understanding is that would only be true if we are actually compiling the cpp files, thatâs different to satisfying a few IDEs, at the end of the day we would still be compiling one large cpp file so if anything changes the whole thing has to be rebuilt.
I would also be concerned but how certain preprocessor definitions and options are handled in headers you would effectively have t duplicate a bunch of this stuff in all the separate headers which runs its own risks. Obviously there are clean ways to break all this up but again thatâs not a small amount of work it will take time.
If Iâm honest I think there are some shortcuts/hacks where you can make the compiler treat each file in the unity build as if it is itself is the unity build. So it will still take time but if itâs just the unity build style that is the issue for some IDEs then I think it would achieve that goal.
Just to add a few other (potential) advantages:
CMAKE_UNITY_BUILD=On is currently not possible without also excluding JUCE from the Unity Build.CMAKE_UNITY_BUILD=On any faster/slower than JUCEâs own Unity Build solution? Perhapsâs JUCEâs CMakeLists.txt could set this by default if it is any faster.In theory JUCE should be faster as it should group closely related files together which overall reduces the amount of parsing required, as many of them will require the same headers.
Donât forget we need to support the Projucer too.
I get there are benefits but only with costs too. If we switched JUCE away from unity builds we would suddenly have a lot of people complaining about the overall build times. Most users shouldnât be changing JUCE source very often (ideally at all) so incremental builds shouldnât be a big issue for most end users.
Depends on what you change.
If you changed juce_somefile.cpp, a rebuild of that module would have to happen. That wonât change with my suggestion one way or the other.
However - if you modify juce_button.h and didnât modifly juce_component.h, each module/unit would hit the #pragma once in juce_component.h, and stop there. There isnât a need to reparse Component since the compiler knows for sure it hasnât changed in all of the current contexts, which it doesnât now.
That shouldnât change.
To do that, you need to add another header file, config.h and have all headers in the module include it. That is a requirement and has to be done for code to not break.
That is also how it works in pretty much all C++ code BTW, this isnât special to JUCE. You have one header of that module with all the commonly needed preprocessors.
No, you wonât have to duplicate anything. You will add a bunch of #include config.h, and that will have an include guard.
All these stuff is really how it always works if you open any C++ codebase in the world (except for JUCE).
No need for such hacks. Iâve already shown in the discussion linked above that this is trivial to do. Hereâs an example for juce_osc:
(This is from 2 years ago, but Iâm happy to send an up to date PR focusing on simple modules like that).
This change is really super super simple. You just add includes, add one config.h file for the common stuff, and remove the global module header include.
The only module where this would take some major work is juce_core since thereâs a bunch of system stuff there that would be need to be moved into their headers, but for all other modules it is something that can be done by the end of the day for the entire codebase.
