This thread feels like it has the potential to boil over.
There are lots of valid criticisms of JUCE here, but please don’t let that spill over into criticisms of each other.
2 Likes
I hear the frustrations and I don’t disagree with all the points.
Unfortunately whatever the JUCE team spends its time on it seems someone is going to be unhappy. @Toddler-Boy you’re clearly unhappy that we spent time on AudioWorkgroups, others seemed unhappy that we didn’t have them in fast enough 
. Obviously AudioWorkgroups isn’t just for AU plugin clients it’s for macOS and iOS Applications/AU hosts too. I don’t think it’s too hard to imagine how a scheduler could improve things by knowing this information.
At the moment (and certainly over the last few months) the team have been leaning towards new features in gearing up for JUCE 8, and at some point this balance will need to shift back to bug fixes.
@Toddler-Boy maybe you’re right, maybe there are more bug fixes we should just accept. However, even when I write them internally and I think I’ve covered all angles, conversations arise that make you realise it’s rarely as simple as you may have originally thought. There are some I’ve done following forum requests that I’ve had to consequently revert because they’ve caused issues for someone else, thankfully before they made into master! Considering not just all the platforms, but all the legacy behaviour that has gone before, that there will be bugs which have now embedded themselves as “features”, and that if we merrily accept something that later turns out to cause us as much headaches as so many decisions that have gone before us we’ll also have users complaining (and more work on our hands).
Now do we always get it right? no absolutely not! we’re not anything special we don’t get it right first time, and not just because of a variable name or formatting (although those things are an important part of the library) but for reasons such as those I’ve highlighted above.
Imagine that we do accept all pull requests, would it be more or less work for the current team? I think it would undoubtably be more work, even if only a small percentage of those PR’s come back to bite us, they will come back to bite us in one form or another. Our own bug fixes do so why wouldn’t those submitted by PR’s do that too!
OK but what if we only accept them from a few particular forum members, maybe the percentage of those that return to cause issues is less? but it’s not going to be zero. What about all those from outside that group? How do we decide who is and isn’t on the list? how does someone get to that status? I could imagine other users will moan that the list of people whose PR’s are accepted is arbitrary, that they fall into some elite category and the only way to get there is years of forum posts. What about those who don’t use the forum? we also get emails and issues on GitHub with bug reports and feature requests. Whatever we do some line has to drawn somewhere.
I’m not suggesting these ideas are bad just highlighting that there are considerations to be made I’m not convinced it’s quite as simple as it’s made out to be.
Here’s the way I see it, if the JUCE team was one person, would we say the problem is that PR’s aren’t being merged or that the team is too small? I would expect most people could agree (maybe I’m wrong) that the real issue is the team is too small, how can one person manage all of that! In my opinion that’s the issue we have right now. The workload for each person on the team increases faster than they can complete it, thus there ends up being a long list of bug fixes we all have on our todo list that we want/need to get done. However small each one might be, each one does require time and effort.
So hire more developers, yes that would be nice but we also need to make sure we only commit to that if we can afford to do so. Anyone can go and look that we’re not doing badly but we’re also not sitting on a gold mine of cash. The team has been steadily growing since it was acquired by PACE, and I’m sure more developers will come in future, but it takes time.
To be clear, I’m not suggesting that there aren’t improvements to be made, there’s always room for improvement. I’m also not suggesting we shouldn’t accept more PR’s I think there’s probably a better balance to be had. What I am saying is that all solutions have compromises, so I’m not convinced that things are always as easy as they are sometimes made out to be. The JUCE team is aware that there are bug fixes that need attention, as I said a balance towards these will change in time and I think that will also be good opportunity for the team to review any processes.
@yfede I see your point but bug fixes need time on develop too, they can have negative consequences just as any feature might, we need an opportunity to change, reassess, or back out of them. We try to do this as little as possible on the main branch. I would also be worried if we cherry picked commits that then when develop is merged into main it’s a possible source of new bugs as main will end up in a different order to develop. At the moment just about everything on main has had some time on develop before it made it to main.
6 Likes
Yes, but for these rare cases where it doesn’t cause problems (crashes, rendering random garbage, etc.) and the fix is only ten lines (or less) of code, wouldn’t testing the fix for 10 minutes and then merging it, be a lot quicker than arguing about it?
Sometimes code needs to be fixed, even if it inconveniences other users. There are ways to do it, by adding a new function and declaring the old one deprecated. You’ve done it before.
It seems that all contributions are seen through the same lens: it’s all unwanted, additional work and thus gets ignored. The JUCE team generalizes. All pull requests are treated the same, even though they differ greatly. Some are only one line of code or not even that. Just a “+1” or “-1” added.
I can fully understand and appreciate that pull requests that touch multiple files, adds/remove dozens of line of code, etc. can quickly become unwieldy and testing takes more time, etc., but for these simple one-liners, off-by-one errors, or even just typos, it would usually only take minutes to test, verify and then merge.
Maybe the JUCE team can take turns. Each member looks at one tiny PRs per day/week makes a decision, then actually responds to the PR (on GitHub) and either rejects (with a short reason) or accepts the PR.
Right now, the PR sub-section is a graveyard, and it looks to me like that is maybe intentional to discourage new PRs. It seems “I don’t need to bother, they won’t accept it anyway”. If that is the goal, then just turn off PRs for the public repo and be done with it.
Here is a helpful action that does just that:
This would send a very clear message.
Instead, you make people jump through hoops and sign papers, just to ignore them anyway.
Let me start of by saying, you’re right there are a number of bugs that we need to address - I’m not going to question that there will be bugs that exist that probably should just be merged. However, maybe I’m not as talented as you at recognising which ones will turn out to be the ones that work without giving them at least a fair amount of attention or writing regression tests for them to make sure we don’t make the same mistake again.
I agree we can do better. I’m not sure in the case of a library it is always better to do 10 minutes of testing then merge. On an app or plugin, absolutely get that fix in, hey add some crash analytics in too, can we see if users are using this feature, no - scrap it. That kind of thing I’m finding much harder for a library, again though I’m not saying we can’t do better than we’re doing, I’m also not saying there won’t be examples where you’re right.
This might sound reasonable but it depends on the “inconviencice” you’re talking about. If the bug is something that impacts a very small percentage of users in practice and/or there are workarounds or some way to avoid the issue, but the “fix” turns out to impact a greater number of users in some other way, especially if it’s hard to notify users of it, then I’m less certain - however to some degree that is what the develop branch is there for.
It’s not intentional to discourage new PRs. I’ve commented on and closed (effectively accepted) a few just recently. We do however have self hosted CI that runs on our own instance of the repo so even when a PR is accepted it’s not normally merged straight in to the public repo as we would prefer it at least runs on our CI. That isn’t to say that the PR’s aren’t useful or good enough.
The amount of code that is touched does not always correlate to the impact something has. I think the example I was referring to was probably just one or two lines of code.
Typos in comments, or probably even the odd string, should probably be a no-brainer, somehow I don’t think that will be the bulk of the problem though. If it was I don’t think there would be the same level of complaints.
We have been doing some, I closed two last week, I have some more I’m working on too. I know it’s not much but these were largely no brainers. Even then frankly I would like to make sure we’re adding tests so we don’t regress again.
1 Like
@Toddler-Boy I also just want to add that I (and I know the team) appreciate the feedback, and I in no way want to come across as dismissing yours and others genuine concerns. I get it, it’s frustrating that there are bugs that have been outstanding for quite some time, and that you have to keep your own fork just to fix those bugs, I’ve been there! I do understand the pain and frustration. For what it’s worth I’ve found old one-line PR’s of mine that haven’t been merged (and still aren’t). The team is small and we’ll need to put some focus on bugs again soon, I would also like to see a focus on improved testing so we can gain more confidence in fixes. I promise we will get there but I’m afraid it’s not going to happen over night.
1 Like
It feels as though “the team is small” is often used as a self-deprecating mechanism to bring discussions to a close, but when I suggested charging those of us more that can afford to pay more to help fund a significant expansion of the team so we can have the best of both worlds I was given plenty of reasons the team feel the team should be kept small. So unless the development of major new features are put on hold for quite a long time (which is clearly not the case with what’s been discussed with Juce 8), what’s ever going to change here?
1 Like
@anthony-nicholls I get it. Regression tests are great and all, but unfortunately that is not always possible (or very hard to pull off), since the more annoying bugs we’ve found (and fixed) are within the UI part. Doing any testing there means writing a component that shows the bug, taking an automatic snapshot of it, and comparing it to a known “good” version. And even that isn’t always working, as in the case of “garbage” being rendered (e.g. opaqueness gets handled incorrectly if the parent(s) have an alpha != 1.0), sometimes you get “lucky”, and the garbage is not there (it renders whatever was in the buffer at the time, which could be nothing = black) and then the regression test “passes”, even though in reality it failed.
For some (usually UI-related) bugs, you have to go the old-fashioned way, fix it, and verify, with your own two eyes and reason, that the bug is fixed.
That’s why @RolandMR created that repo with all these bugs so that we can demonstrate the issue. We also often offer a fix. We use these fixes in commercial projects with hundreds of thousands of users.
1 Like
This is also an issue. We use the “develop” branch because there are often essential bug fixes that are only on “develop” and get merged into “master” several months later. We can’t wait that long and tell our customers “well, it’s fixed on the develop branch, but not on the master, so you will have to wait”. Our customers don’t care. They want a working product.
Whenever somebody reports a bug, one of the first questions is, “did you try the develop branch? Maybe it’s already fixed there”. So it’s damned if you do and damned if you don’t.
Officially, only the “master” branch is supported, but if we find a bug, we’re directed to the “develop” branch.
New features (like the AudioWorkgroup) should have their own branch, be tested thoroughly by those who want to use them, and only merged once sufficient feedback has been received.
2 Likes
(post deleted by author)
(post deleted by author)
(post deleted by author)
I agree with Anthony’s points. My two bucks:
As an open-sourced project, JUCE is not obliged to do things exactly asked above by someone. What I want to state is that his attitude of asking these things above won’t help him persuading those he wants to persuade. At least, if there’s a product manager for JUCE, it is someone else. Product managers think hundreds steps further than a customer too angry to think why the product manager made these decisions.
I admit that I have the same issue. As an input method developer (for macOS) for almost 2 years, my anger is towards Apple. Their input method kit in macOS has lots of bugs not fixed in more than a decade, and they never give a **** to those bug reports related. But it might not be an attitude problem but the lack of human resources… too much possibilities, and our individual abilities of imagining things are limited.
A similar thing is Dorico: I started using Dorico since the dev beta of its v1. In early versions I found some limitations in Dorico, and I did complain some long-term issues with Daniel. However, till now, by seeing all major Dorico releases in these years, I am afraid that every of its feature implementation has been considered and planned years ago.
This leads to a new topic: How to interact with product managers whose product is not runned (nor managed) by you, which runs out of this thread again.
@anthony-nicholls BTW, if you want an easy way to find the related bug reports for the repo I’ve shared. Use the forum search for https://github.com/FigBug/juce_bugs. Then sort by “last post”. This will give you a pretty good idea, with a description, screenshots, etc.
There you can also see if somebody from the team replied and potentially has already fixed the issue.
1 Like
Honestly I would want avoid a test like that as much as humanly possible. If I was writing a test like that I would want to be in control of the buffer that was passed in to ensure I could test it with a greater deal of confidence. To do this you generally need some form of dependency injection in order to decouple things. There’s lots of parts in JUCE where doing this is indeed difficult (or currently impossible) but in some cases with a little attention we could make changes to make testing easier.
Recently I’ve been looking at ways to do this with the MessageManager, currently it’s basically impossible to do tests with it. I’m experimenting with a branch at the moment that allows us to write tests and it’s already found a fairly significant bug with the MessageManager::Lock which was ironically introduced fixing a small “bug fix” reported on the forums. I say bug fix in quotes because really it was a feature request in disguise, but it seemed reasonable. I already tried fixing it and quickly that fell apart into chaos hence needing automatable tests!
Anyway I was referring to the fact that I wanted to add tests to the specific PR’s that I was looking into, and that it would be nice if this was also easier for users too, then they can submit them which in turn increases confidence in the fix, as well as help communicate what the issue really is (as I expect the examples you and @RolandMR have provided do 
).
However, I respect sometimes (esp. with GUI related bugs, even more so for those on specific OS version etc.) you just need to load it up and view - and we absolutely do that.
Indeed I guess we hope that the effort for the reporter to check out the latest tip of develop to reproduce the issue they’ve already managed to reproduce locally is smaller than the effort required for a member of the JUCE team to try to reproduce the issue and then be stuck not knowing if they can’t reproduce it because there is some missing step / key ingredient or it’s just fixed on the latest version they are working with. It’s not uncommon that I see users reporting bugs that turn out to be on older versions of JUCE (i.e. not the latest tip of master or develop), so asking users to check out develop seems sensible, no?
In my professional experience, most of the time I’ve managed to release products from master and only switch to develop for short(ish) periods of time, and whilst on develop we’ve tended to stay on a specific commit. Albeit normally with a fork of JUCE (although not always!).
Now obviously you being on develop and testing and reporting back on the forum is really helpful and we appreciate it so I don’t really want to discourage it either. I am however interested to know which bug fixes are currently on develop that you need?
Much appreciated thanks for sharing.
@Verbonaut I’m sorry you feel that way, I’m not intentionally trying to close the discussion, I’m just trying to give an honest opinion. I think your suggestion of charging more for those that can afford it may well be sensible but I don’t think it’s something we could just commit to in response to a forum post!? I would expect a review of prices for any new release of JUCE. That being said I’m not sure that a significant expansion of the team (especially over a short period) would fill me (thinking as a customer here not a developer of JUCE) (or lots of customers) with joy. Imagine just how much of a halt JUCE would come to if we did that! The hiring efforts would be enormous, and then all the onboarding. Like all solutions a larger team will have its compromises too.
I already stated that I agree that the team does need to increase in size, although that is just my opinion, but I also think there are improvements that could be made to our processes that would help, none of this will happen over night though. Having joined the team relatively recently it is my opinion that we would be best getting JUCE 8 out first and then concentrating on some of these changes. I realise you and others may disagree but I’m not sure there is a lot I can or should do about that.
So let me try to summarise, am I right in saying the main themes here are
- AudioWorkgroups is a feature some users want, others don’t
- Bugs happen on the develop branch
- The develop branch has sometimes taken too long to be merged into master
- Some users need to use the develop branch for production builds because it contains important bug fixes or features
- There are long standing bugs which users have supplied fixes for that could/should be merged into JUCE
Without diving into all the possible solutions we’ve discussed for any of these have I missed any of the important general talking points? Maybe something along the lines of…
- The balance of work is currently weighted towards new features rather than bug fixes
1 Like
Currently, there is nothing on develop that we would need in master, but right now there is a bug in the SVG parser, the fix is on develop. Since it’s a brand-new bugfix, we won’t expect it on master right away, but if there are multiple users confirming the bug is fixed, we would like to see this merged ASAP.
We have our own fork, and it’s based on the develop branch, and it gets auto-merged once per hour (or so). Switching to master is simply not an option for us, unless we want to fork and auto-merge that too, so we can switch back and forth (which means recompiling the whole project, etc.)
For all practical purposes, we have to stay on develop and be vigilant. If everybody stayed on master, you wouldn’t get any feedback at all and fixes would go completely untested.
1 Like
Understood thanks for sharing.
Just to give the team something to think about: You all know the principle of “death by a thousand cuts”. To me, this is comparable with bugs and new features. Focusing on features and (almost) ignoring all the little annoyances (bugs) makes everybody suffer this death by a thousand cuts.
Multiply that by the number of developers, and then multiply that by the number of their users.
Now you have to ask yourself what is worse: not having a particular feature or having dozens (if not more) of small bugs?
To me, the choice is pretty clear. Having potentially millions of people suffer so a few can have a new toy to play with, is a pretty easy decision for me to make.
The same applies to the reputation. Would you rather have a few developers complain about a missing feature or would you rather have ALL developers complaining about dozens or hundreds of bugs? What is more work? What affects the reputation more?
The JUCE stated the reason for not adding “CLAP” as a new plugin format: it’s not available on all platforms, and the potential number of users/developers is small. It’s not worth the risk and development time.
Why isn’t the same standard applied to AudioWorkgroups? It’s the exact same problem. Only macOS/iOS and not a lot of demand. Obviously, I’m not suggesting removing that feature now, but the time could have rather been spent on fixing 5-10 smaller bugs (if not more).
1 Like
I’m not sure I agree it’s all the developers, those issues are generally not fixed by small one-liners (topics that come to mind are text rendering, unicode support, blurs/gradients, etc.) however point taken and thanks for sharing.
1 Like
@Toddler-Boy: how likely is it that you could de-couple your regression tests and share them with the community? Seems like some of us might have the resources to integrate this with existing CI workflows, giving us some semblance of a tool to catch regressions.
Catching up on this thread, it really seems to me the logical step is to add automated tests that can be used to map out the impact of changes/bugfixes happening on develop … so any chance of a community organized effort to contribute this to the JUCE project, as a whole?
Unfortunately, the chances are pretty low. We had to make a few changes to our products to accommodate their testing, e.g., upon preset load, all parameters that rely on randomness get changed to something that doesn’t rely on randomness anymore because these tests would otherwise always fail.
We also made some changes, so whenever setStateInformation gets called, we can check for special “magic” strings in the state to do some remote control of the plugin.
This only happens for the “development” build of our plugins (basically release optimizations, but with debug information turned on and a few extra blocks of code here and there).
Then, we can use the command-line parameters to trigger specific behaviors for the stand-alone version.
This is all very product-specific and thus not easily shareable.
The comparison of the wav-files is trivial. We export as float-wav to detect if a preset is outside the normalized range. We check if the volume is below a certain threshold (effectively silence) and if the float difference between two wav-files is above a certain error margin. We then keep a tally of how many samples are failing, how far they are off, and the mean and maximum deviance. We can then set thresholds for “errors”, “warnings” and “OK” and print those to the console with the appropriate colors.
That part is completely stand-alone and could be shared, but it wouldn’t be of much use unless you already have the ability to create comparable, deterministic wav-files in the first place.