About git submodules

This thread is intended to continue the discussion about git submodules started in the following forum thread: About sharing Juce under GPL

@daniel After more investigation about the submodules and their caveats… I am once again changing my opinion about using them, since the amount of complexity they add to the procedure of clone and build a project is disproportionate. Let me add more details below:

I recommend the following read about git submodules, I have learned a lot by reading it: https://medium.com/@porteneuve/mastering-git-submodules-34c65e940407

Once again reminding everyone about my use case:
-I use Projucer as platform-independent project management tool.
-I use various Juce modules for my project (GUI, OpenGL, audio)
-My project uses Juce under GPL license, which brings the main issue:
-I want anyone who clones my project to have access to the GPL version of Projucer, and not have to deal with enforced splash screen and usage analytics.

Now, this last point might already be well-known for all of us Juce users, but for someone who just wants to clone and build a project, asking them to investigate or be aware of the Juce licensing matters, modifying the necessary code of Projucer and re-build it in GPL mode, etc… feels detracting to any user who just wants to get things done quickly (which nowadays is probably most people).

Until Juce 4, I just had “Download and install Juce” as step #0 in my build instructions, since it was simple enough. Now with all these license matters, I want to add a shortcut for any potential user of my project, by providing access to a Projucer which source is already modified to build in GPL mode, as well as a pre-built GPL Projucer for Windows users.

My initial intention was to add the Projucer code and binary, as well as the necessary Juce modules for my project, directly in a folder within my repo. Simple enough, ready to be used, and if any user wants to dig deeper in Juce, they have the link that points them to the full installer.

However, @daniel recommended in the original forum thread to use a git submodule instead, pointing to my own fork of Juce which includes the necessary modifications. This way, any user would be free to pull any changes they might want from the original repository of Juce, etc (I am not an expert in git or in submodules, so please correct me if I’m wrong). However, with this solution, I think I would still remove the unnecessary submodules, the whole examples folder, as well as all the extras (except for Projucer), to make the fork lighter (it takes me more than 3 minutes to clone the whole Juce repo).

All this into consideration, I feel once again inclined to just add a folder in my repository, after reading about the complexity and all the potential problems of git submodules, in the aforementioned link.

Does anyone have any advice for my use case? Which of the above options do you think is more appropriate? Or would you do it following any other method? Does anyone hold a strong opinion in favor or against submodules?

I will appreciate any insight you can offer in this discussion :slight_smile:

Submodules can be a bit of a pain, but they’re worth getting to grips with for anything but the most trivial projects. Copying files into folders and committing any changes yourself really isn’t a scalable alternative, and I think you’ll regret that in the long run.

1 Like

HAndling submodules by hand is a nighmare, use a tool like Sourcetree to do all of it for you.
But please, don’t do like Steinberg and put every folder in a submodule (VST3…). It feels like a newbie learning about design patterns and overdesigning their architecture.

I am using git subtree for including Juce. I remember that this was for strong reasons, but I forgot why, though…

git remote add -f juce git://github.com/WeAreROLI/JUCE.git

Update to a tagged revision:

git subtree pull --prefix=source/vendor/juce juce 3.1.1 --squash

Update to the newest revision:

git subtree pull --prefix=source/vendor/juce juce master --squash

I case it’s of interest, our approach at ROLI and Tracktion is to have one big top-level repo containing all the code, but for libraries which need to be shared externally (e.g. juce and other special cases), these go in submodules inside the big one. So they’re really essential, but yes, definitely use as few as you possibly can!

I do the same as Jules pointed out. Especially as your code relies on JUCE, I don’t see a point not to do so.

To get your users to have the GPL mode, a simple solution is to check in the generated project files (vcproj. and xcodeproj). Remember, they were generated with your GPL-enabled Projucer version, so as long people just compile your code, all works fine. As soon as they start to add files etc, they use their Projucer, and then it is up to them to figure out, if they are still covered by the GPL license. So I can’t see the problem here.

For these three minutes, did you do a shallow clone or the full with history? To put a price tag on it, if I run:
git clone --depth 1 -b develop https://github.com/WeAreRoli/JUCE.git
it takes 5 secs here, the repository takes 77MB:

Didn’t hurt too much… I also searched for a way to add examples to .gitignore, but seems that this is not possible. Also given these numbers, is it worth it?

Thanks for the link, there were several things, I didn’t know before. However, the introduction sounds like a scarecrow, looking for arguments:

Every time you add a submodule, change its remote’s URL, or change the referenced commit for it, you demand a manual update by every collaborator.

This is only a problem, if you add the submodule after somebody already cloned it. You are just starting, so is there a chance, somebody already cloned it?

Because lifecycles are separate, updating a submodule inside its container project requires two commits and two pushes.

This is only happening, when you change stuff inside the submodule, here JUCE. How often will that happen?
And you always see git status showing if there is modified content inside the submodule. But I do like his suggestion on turning on additional info on submodules!

Removing a submodule requires several commands and tweaks, some of which are manual and unassisted.

Again, does your project work without JUCE? How likely is it that you want to get rid of it?

These are all good insight, but I think they don’t matter too much in our usecase.

But the bonus is, if something used to work and stops working, I can go back and forth in JUCE history to find the problem. Either to blame the guys ( :wink: ) or to find ways around.

And if you cloned your fork, by adding the original as additional remote, you can pull their bugfixes on your copy at any time:

git remote add juce_roli https://github.com/WeAreRoli/JUCE.git

and later

git pull juce_roli develop

to get the latest juce commits. Your repo will show then “Updated content in submodule JUCE”, and if you want to have your repo pointing to that new version, simply commit your project.


Thanks once again for the additional comments! As you may have figured out, I am not that familiar with git, I only use the “basic” functionality of it and don’t have experience with things like submodules. I will re-read all the comments here to make sure I understand and absorb them correctly.

One particular thing I didn’t know is the shallow clone. I will certainly try it out and see if my super-slow clone was because of that :slight_smile:

1 Like