New: stable + unstable JUCE branches on github

[timur]

Hi all,

Thanks a lot from the JUCE team to everyone here who provided feedback after the release of JUCE 4.2. This is extremely valuable to keep growing and improving JUCE. I believe that it’s the greatest strength of JUCE to have this strong, vocal, and active community! Many of you brought up great ideas about how to improve our workflows. One of the most requested details was the desire for having both a stable release branch and an unstable bleeding-edge branch on github (instead of the single “newest tip” that we always had in the past). So we thought about how that would work and came up with something. From now on, there is a master and a develop branch on github. master is now our new stable branch, and develop takes the role of the “newest tip”. Enjoy!

Below are a few details about we’d like those branches to work. Please view this as a first iteration–of course we can adapt and change this to whatever works for everyone. Please provide your feedback and let us know if you spot anything that wouldn’t work for you!

1. There are always exactly two branches on github: master and develop.

2. master is the most current stable and tested version of JUCE. You can rely on master for building and releasing software to your customers. master only changes if there is a new stable release or a critical hotfix.

3. develop is the bleeding-edge tip of JUCE. You can use develop at your own risk to check out new features and improvements in advance, and get quick fixes.

4. develop is always ahead of master or equal to master.

   

5. New features are tested by us and then pushed to develop. The JUCE community can test and integrate those features and provide feedback to the JUCE team.

6. Bugfixes, improvements, etc. are also pushed to develop.

7. Releases are defined by selecting a commit on develop that the JUCE team unanimously agrees to be stable, fast-forwarding master to this commit, and tagging it with a version number.

   

8. Hotfixes are committed to master. Then, master is merged into develop.
   (Hotfixes are critical fixes that need to be released immediately, such as fixing bugs that compromise the security of our users, or that break a headline JUCE feature for most or all users.)
   

   

9. The dowloadable version on juce.com is always equal to master. This process is automated. (we didn’t set that up yet but we will asap)

10. All breaking changes on master are communicated in advance. If non-obvious code changes from our users are necessary, we’ll do our best to provide a migration guide or other documentation as appropriate.

Please let us know what you think!

Edit 26/04/2016: point 8 about hotfixes used to say that develop is rebased on master. It turned out that could break people’s local branches, so now we’ll merge master into develop instead.

[kunitoki]

good monday news… finally !

if you allow settle down new features in the develop branch where we can test them in advance then the overall solidity of a release will be better !

[bld]

I voted yesterday for release branch also (in the topic: Plugin binary size (4.2 = bloat)

We can live without release branch but please at least when you do (ad point 7) new master commit tagged as new version, mark it first as RC (Release Canditate) - 4.4RC in this example.

This way you send signal to us that you think that this particular develop commit is stable enough to be new version.

And then we can do more checking on RC and give you feedback if we also agree that it’s stable. Because not all of us will be able to follow the develop branch all the time and check it constantly.

You can simply remove the “RC” from version tag after some time of silence where no bug is found. And we can avoid the problems of using the not-so-stable release in our products.

Cheers,

[timur]

Hi,

RC’s are a good idea in general, but that’d be yet another (third) level to add. It feels like we should not introduce too many additional complications at once! I think that distinguishing between master and develop is already very helpful.

And yes, the idea is that you’d be checking the develop branch. It is now essentially exactly the same thing that master was until now (what we refer to as the “newest tip”). It seems that most people here are using it.

[bld]

But tagging RC on master branch first and then rename the tag is not any third level. You still have two branches only… just my suggestion. Anyway thanks for the splitting - it’s big improvement.

[yfede]

Thank you, this is great news!!!

One question, though: in case of a hotfix on master, why do you specifically intend to rebase the whole develop on it, rather than merge the hotfix added to master on top of develop and continue from there?

I’m sure you know that rebasing develop will break anybody’s repository that happens to be using JUCE develop as a submodule.

Plus, I guess that the hotfix in question will on most occasion be a cherry-pick ‘backport’ of something that has already been tested in develop, thus merging it in again from master will actually be a “no-op” merge.

I know that this breaks the strict linearity of the history, but thanks to the merge at the top you will make it clear that the two branches have not diverged and never really will.

Just my two cents and I agree that time will tell, for now thanks for having finally added this

[timur]

I’m sure you know that rebasing develop will break anybody’s repository that happens to be using JUCE develop as a submodule.

Yeah that would be an argument for merging and against rebasing. (the argument for rebasing is the clean linear history).

However at least from my experience if you just rebase a branch on top of another, even though technically you have to do a git push --force, overwriting the remote, in this specific case people’s tracked branches typically don’t break (git is smart enough to figure out it’s just a rebase) and they still can pull normally. At least it works here for us. So I’d prefer the rebase unless people’s branches would actually break (anyone has experience with this different from ours?)

[jonathonracz]

Awesome step forward! Glad you guys are directly responding to feedback from users.

I guess I’m a little shaky on why you guys are insisting on a clean project history via rebasing when IMO periodically merging the develop branch back into master gives a clearer picture of the changes from version to version of master as well as avoiding *potential* breakage for end users’ repos. I don’t think I’m aware of any other major project that manages branches via rebases like this… but I’ve never done project management for a company, so I might be missing something.

Also Timur, you missed an “n” in “dowloadable” in point 9

[yfede]

people’s tracked branches typically don’t break (git is smart enough to figure out it’s just a rebase) and they still can pull normally

Unfortunately, pulling is not the only thing one does with branches and commits.

JUCE is a submodule in my project, hence the main repo knows the hash of the JUCE commit that it should checkout when the project is cloned.

Let’s assume that I am doing some development using the develop branch, on my Mac.
Then I go on vacation for a week, and when I am back I wish to test the state of my project on my Windows machine, thus I clone my project there.

At this point, if the develop branch has been rebased in the week I have been away, the newly cloned repo for my project has no chance of finding the JUCE commit that it needs only by its hash, and unfortunately the hash is the only information that it has because there is no tracking branch yet.

More than that, at that point I’ll have to manually find the commit to checkout that restores the situation as I left it, but during a week JUCE has probably advanced fifty or so commits, and I may not even remember what’s the description for the commit I need.
It’s going to be unnecessarily time consuming and it may also result in noise in the forum.

(it may sound hypotetical but clone is exactly how we pass projects from one machine to the other during development)

P.S. don’t take mine as a complaint but rather as constructive feedback, I still appreciate very very much the publishing of the two branches

[timur]

Cool, thanks for the feedback & for sharing some knowledge about git. We’ll discuss whether merging is perhaps indeed better as rebasing. We do like a nice and clean linear history. But as we now know, not breaking your stuff is kinda important, too At ROLI, we also use submodules, so we can easily check how it behaves for your usecase.

[koentanghe]

Thanks so much for doing this, guys!
This is indeed great news, and highly welcome!

Personally, I typically use merging instead of rebasing. I don’t mind the extra merge commits in the history (I actually find that way of working more transparent).

What could also be good, perhaps, is to use feature branches for bigger changes (but maybe you do that already in another place internally).

It’s very nice to see a bit more structure coming into Juce development. I guess now that the team is a bit bigger than “only” Jules, this had to come up at some point also internally (not only for us). So, very happy about this!

[timur]

I just tried the case with rebasing + having JUCE as a submodule, and yes you guys are right, that would make your local repo go out of sync and you’d have to manually change the commit the submodule is pointing to every time it happens.

So yes, to avoid that we’ll be merging master into develop instead. I edited the original post above.

Thanks everyone for spotting this!

[myhrman]

Excellent, great work listening to these requests!

Did you consider using the already well-documented Gitflow workflow? There’s great Gitflow support in most clients, allowing you to do basically do “one-click” operations for common tasks, like finishing features and merging develop back to master, like being discussed above.

Gitflow Workflow | Atlassian Git Tutorial

[timur]

Yes, I am familiar with GitFlow, I used it at another job before I joined JUCE/ROLI and it worked very well there.

[pflugshaupt]

I’m sorry but I have to vent a bit of frustration about the whole develop/master branch thing. I’m about to release a plugin and would like to use a stable JUCE version but in my experience the merges to the master branch fail to be really stable. This is for sure true with the latest 4.3.1, but I remember things were similar with earlier version increases. Basically I feel there is not enough time for develop changes to fully mature. To us “consumers” the merge just happens at random times and new patches just slip in. A new develop patch might have been added just hours before it is labelled stable this way.

For instants I cannot release with 4.3.1 because of problems with both the AAX and VST3 wrappers. To me it feels the same like before you had the develop branch with an added layer of pseudo stability confusion provided by the master merges.

I am happy to be on the develop branch for most of the time and help try out new features, but it feels to me no new features should be in a stable release until they have been on the develop branch for some time to get a proper workout and have possible regressions pop up from your users. The more reliably stable the master version steps are, the more useful the whole system gets. As of now at least for me its useless and it is going to annoy new users thinking the master is always stable. I don’t know how something like that can be achieved with git, but I’m certain there is a way.

[jonathonracz]

It seems like part of the problem is that none of the commercial users really use the develop branch (with good reason), so none of these bugs are found until it’s arbitrarily merged into master and everyone complains.

I’d be curious to know what the internal unit testing situation is - what should be (and I hope is, but doesn’t seem to be) happening is each version is fully compiled and unit tested with a majority amount of coverage with every compiler and OS supported by JUCE. The fact that things like compiler warnings in VS2013 or some other supported compiler get into customer branches confuses me a lot.

That’s not even asking that much. A 2013 Mac Pro could easily run VMware ESXi with every supported OS and be triggered to do this automatically on git commits while quietly sitting in a corner. I’ve done it, and I’m sure others on this forum have too, so ROLI should be able to especially if JUCE is a commercial library.

[kunitoki]

in my juce fork https://github.com/kunitoki/JUCE/blob/develop/README.md i already integrated it with travis and appveyor. that is limited to compiling and executing the unit tests but when i will publish the juce automation framework i’m working on, it will be possible to automate an application quite easily and also automate juce components for regression testing in jenkins slaves too (or locally).

[yairadix]

It seems like part of the problem is that none of the commercial users really use the develop branch (with good reason), so none of these bugs are found until it’s arbitrarily merged into master and everyone complains.

I can assure you that some commercial users do use the develop branch, with good reason (bugs are sometimes fixed only there).
However here at SR, we also have to maintain our own branch with various fixes and changes on top of the develop branch, although we really do try out best to report bugs, investigate them and offer fixes and have them incorporated into JUCE. At the present time our changes are mostly for macOS OpenGL crashes, RTAS side-chain support and macOS graphics performance issues. I know of other companies in similar situations.