Int vs float rectangles with scaling: a proposal for Windows/MacOS consistency for JUCE 8

[fuo]

Hi there,

I have been trying out the new Direct2D backend quite a lot lately, and was really impressed at the performance and integration within the Juce framework. @matt did a fantastic job there!
Now the performance is on par with the current Mac renderer, if not better.

The main thing I focused on, and did a lot of reports about, is the behavior when scaling the UI (AffineTransform::scale). This is the preferred method when resizing vectorial UIs, and it makes a lot of sense too with fast renderers (no need to buffer to images, path and gradients are cached, etc.).

The current Direct2D branch still has some issues when dealing with scaled int rectangles, but I assume most of them will be solved by the time JUCE 8 is released. I will focus on the behavior of the openGL and software renderers as I assume this is the current “behavioral” model for the direct2D one in this regard.

So, even with the openGL or software renders there is still a visible difference between float and int rectangles: float rectangles are much more consistent and stable (edges are less bouncy) when changing scale (especially visible with small scales), and as a consequence fine margins between adjacent rectangles stay visible much longer when scaled down compared to int rectangles. You also get inconsistencies when mixing int and float rectangles (unaligned edges a certain scales, etc.).
This situation also affect components, which bounds are always int rectangles. You can fill them with a float rectangle rather than an int one (or a simple fillAll) to get that same improved consistency with low scales, but that is only an option for custom components of course.

Now, on MacOS the situation is different with the default renderer: both int and float rectangles (with the same round values, of course) do behave exactly the same when changing scale, and the same goes for components. In fact they behave like float rectangles do on Windows: they are extremely well behaved and the UI seems locked-in down to the smallest possible scale. This is very impressive and feels nice (which is important for user experience).

Switching the openGL on MacOS gives the exact same behavior as the openGL and software renderer on Windows, with the int vs float difference becoming obvious again, and the UI becoming more bouncy when changing scale.

So here is my proposal:
Please make the Direct2D renderer behave like the MacOS one, ensuring that int rectangles (and components bounds) behave the same as float ones of the same values.
For one thing this would improve consistency between MacOS and Windows, improve scaling stability, and also get rid of many tiny inconsistencies you get when mixing int and float rectangles with a scaled UI.

I think JUCE 8 is the perfect opportunity for this kind of change: keeping the Windows software (and openGL) renderer unchanged would ensure that the rendering (both in terms of look and performance) could stay unchanged compared to JUCE 7 by sticking with it. On the other hand the (default) direct2D renderer would behave much closer to the MacOS one, both in terms of performance and (more importantly) look.

That would be a move for the best for IMHO, and a major version combined with a new renderer is a golden opportunity to take the plunge.

So in short, I believe the Direct2D renderer should use the MacOS renderer as a model, not the current Windows software one.

[chkn]

I think it boils down to the inconsistent behaviour of fillRect(int, int, int, int) on the windows software renderer.

Subtle drawing bug(?) -> fillRect (integer version) scaled

Hi Andreas, its totally normal that with a non integer scaling factor, that calculated pixel positions may will collapse on the same physical position. Example: if the non-scaled coordinates are 2 and 3, they will transformed to 2,5 and 3,75 (with scaling 1,25) and will be part-wise mixed on the same physical pixel 3. The problem described in this topic is that there is a graphic function doesn’t work this way: the int version of fillRect. If you use this function, you should replace it with…

        g.fillRect (10, 2, 2, 2);
        g.fillRect (20.f, 2.f, 2.f, 2.f);

[reFX]

We would never use the int versions of these rendering functions, if it can be avoided. We exclusively use the float versions.

[fuo]

That is a good strategy, but that also means avoiding stock components, including complex ones like TabbedComponent, as most of them either use int rectangles or fillAll, and lookandfeel alone is often not enough to replace all those painting logics.

[reFX]

Then we avoid those as well and write our own. fillAll is fine, but drawing rectangles, images, etc., using integer coordinates is a big no-no for us.

[fuo]

fillAll is just like using int rectangles as it fills the whole bounds of the component, which are an int rectangle.

[chkn]

fillRect (int, int, int, int) should just behave like all other drawing routines.

I think there is a general mathematical problem, which is not specific to juce, when drawing adjacent sub pixel rectangles is that the continuity information between rectangles is lost because the pixel resolution is too low. In the audio sector, this would be combated with oversampling.

And for the special case, if you have adjacent rectangles that are in a scaled environment, you can either use paths which include both rectangles in a merged way (which is the “correct way” to do it, but impossible if more components are involved) or you need a special routine, like fillRectEnlargeToFitPhysicalPixelsIgnoreClippingBounds(int, int, int, int).

[fuo]

Sure, and this particular issue impacts both MacOS and Windows (albeit to different degrees, more on that later), but that is not exactly the problem I am talking about here.
My main issue is with the way int and float rectangles behave differently when scaled on Windows.

Here is an example illustrating these int vs float differences:

class TestComponent : public juce::Component {
public:
	TestComponent() {
		for (auto& c : mFillAlls) addAndMakeVisible(c);
		for (auto& c : mFillRectInts) addAndMakeVisible(c);
		for (auto& c : mFillRectFloats) addAndMakeVisible(c);
	}

	void resized() override {
		auto bounds = getLocalBounds();

		mTextColumn = bounds.removeFromLeft(250);

		auto fillAllComponentRow = bounds.removeFromTop(60);
		auto fillRectIntComponentRow = bounds.removeFromTop(60);
		auto fillRectFloatComponentRow = bounds.removeFromTop(60);

		for (auto& c : mFillAlls) c.setBounds(fillAllComponentRow.removeFromLeft(60).reduced(1));
		for (auto& c : mFillRectInts) c.setBounds(fillRectIntComponentRow.removeFromLeft(60).reduced(1));
		for (auto& c : mFillRectFloats) c.setBounds(fillRectFloatComponentRow.removeFromLeft(60).reduced(1));

		mIntRectRow = bounds.removeFromTop(60);
		mFloatRectRow = bounds.removeFromTop(60);

		mIntRectJunctionsRow = bounds.removeFromTop(60);
		mFloatRectJunctionsRow = bounds.removeFromTop(60);
		mMixedRectJunctionsRow = bounds.removeFromTop(60);
	}

	void paint(juce::Graphics& g) override {
		g.fillAll(juce::Colours::white);

		auto textColumn = mTextColumn;

		for (auto& text : mTexts) {
			g.drawText(text, textColumn.removeFromTop(60).reduced(10), juce::Justification::centredLeft);
		}

		auto intRectRow = mIntRectRow;
		auto floatRectRow = mFloatRectRow;
		auto intRectJunctionsRow = mIntRectJunctionsRow;
		auto floatRectJunctionsRow = mFloatRectJunctionsRow;
		auto mixedRectJunctionsRow = mMixedRectJunctionsRow;

		g.setColour(juce::Colours::black.withAlpha(0.5f));
		for (int i = 0; i < 4; ++i) {
			g.fillRect(intRectRow.removeFromLeft(60).reduced(1));
			g.fillRect(floatRectRow.removeFromLeft(60).reduced(1).toFloat());

			g.fillRect(intRectJunctionsRow.removeFromLeft(60).reduced(0, 10));
			g.fillRect(floatRectJunctionsRow.removeFromLeft(60).reduced(0, 10).toFloat());

			if (i & 1) {
				g.fillRect(mixedRectJunctionsRow.removeFromLeft(60).reduced(0, 10).toFloat());
			}
			else {
				g.fillRect(mixedRectJunctionsRow.removeFromLeft(60).reduced(0, 10));
			}
		}
	}

private:
	struct FillAll : public juce::Component {
		void paint(juce::Graphics& g) override {
			g.fillAll();
		}
	};

	struct FillRectInt : public juce::Component {
		void paint(juce::Graphics& g) override {
			g.fillRect(getLocalBounds());
		}
	};

	struct FillRectFloat : public juce::Component {
		void paint(juce::Graphics& g) override {
			g.fillRect(getLocalBounds().toFloat());
		}
	};

	juce::Rectangle<int> mTextColumn, mIntRectRow, mFloatRectRow, mIntRectJunctionsRow, mFloatRectJunctionsRow, mMixedRectJunctionsRow;
	std::array<FillAll, 4> mFillAlls;
	std::array<FillRectInt, 4> mFillRectInts;
	std::array<FillRectFloat, 4> mFillRectFloats;

	juce::StringArray mTexts{
		"fillAll components",
		"fillRect int components",
		"fillRect float components",
		"fillRect int paint",
		"fillRect float paint",
		"fillRect int paint junctions",
		"fillRect float paint junctions",
		"fillRect alterning paint junctions"
	};

	JUCE_DECLARE_NON_COPYABLE_WITH_LEAK_DETECTOR(TestComponent)
};

class MainComponent  : public juce::Component {
public:
	MainComponent() {
		setSize(referenceWidth, referenceWidth);
		addAndMakeVisible(component);
		addAndMakeVisible(zoomLabel);
		refreshZoomLabel();
	}

	void resized() override {
		refreshZoomLabel();
		zoomLabel.setBounds(getLocalBounds().removeFromTop(30));
		component.setBounds(0, 0, referenceWidth, referenceWidth);
		component.setTransform(juce::AffineTransform::scale(getScale()).followedBy(juce::AffineTransform::translation(0,30)));
	}

private:
	void refreshZoomLabel() {
		zoomLabel.setText("zoom: " + juce::String(juce::roundToInt(100 * getScale())) + "%", juce::dontSendNotification);
	}

	float getScale() {
		return (float)getWidth() / referenceWidth;
	}

	juce::Label zoomLabel;
	TestComponent component;

	int referenceWidth = 600;

	JUCE_DECLARE_NON_COPYABLE_WITH_LEAK_DETECTOR (MainComponent)
};

You can compile it and play with the scale by resizing the window, and see how rectangles behave.
On Windows Int rectangles (and components) are bouncing around, and their 2px margins start to disappear as soon as you start scaling down the window (as soon as 98% with a 100% display scale). By 65% the margins are invisible most of the time.

I also added three rows at the end with adjacent rectangles to illustrate what @chkn mentioned.
In addition to seams (and even overlaps on windows, which are visible becauses the fill colour has a 0.5f alpha), the last row illustrates the misalignments you get when mixing int and float rectangles.

Once gain, these int rectangle geometry problems also impact components filled with fillAll or fillRect, as illustrated by the first 3 rows.

All these problems are not present on MacOS, where int and float rectangle (and components bounds) behave exactly the same (and are well behaved), and can be mixed without alignement problems.

Here are a few screenshots of a selection of scales on both Windows and MacOS, both using the default renderer as of JUCE 7.0.11:

Windows, 100% Display scale, Juce scales: 100%, 103%, 94%, 54%

MacOS, 100% Display scale, Juce scales: 100%, 103%, 94%, 54% :

[fuo]

One additional thing to consider: is it always safe to use isOpaque(true), or even setPaintingIsUnclipped(true) when filling a component with a float rect ?

[stephenk]

It’s great to see this demonstrated so clearly and succinctly.

[fuo]

Good news: it looks like this has been addressed in the current JUCE 8 preview branch for the three windows renderers!

Thank you Juce team!!