Direct2D preview branch

[matt]

@matt

I’ve seen the shifted gradients before, especially when applying transforms to gradients. Those are still eluding me; I don’t fully understand some of the code in the software renderer that positions the gradients (GradientPixelIterators::Linear).

Actually, never mind; there was an offset in the Direct2D code that needed to be removed.

Matt

[matt]

Looks like I spoke too soon; there’s still time to slip in some of these fixes. They should be available once CI is sorted out.

As far as getPhysicalPixelScaleFactor(), could you provide an example of a visible rendering difference between the software renderer and the Direct2D renderer?

The gradients are CPU resources and are cached internally by the Direct2D renderer, so you shouldn’t need to recreate them on a DPI change. Anything cached in the GPU does go away on a DPI change (such as an Image).

Matt

[fuo]

Hi @matt

As far as getPhysicalPixelScaleFactor(), could you provide an example of a visible rendering difference between the software renderer and the Direct2D renderer?

I am a bit confused right now: I tried to build an example with a simple image but now I cannot get anything else than a black image with this branch, even when changing renderer.
I’d prefer to leave this at that to avoid confusion.
I would love to try the last modifications but it looks like they will not make it into the Direct2D branch before Juce8?

I think I also have that issue with setBufferedToImage still, but I did not try to reproduce it in a simple demo code.

Regarding getPhysicalPixelScaleFactor, my use case is simple: I need to generate complex images and cache them, and keep them till the scale changes. The image also requires the correct final scale, and this is all done in the paint method.

The gradient I was referring to in my example is derived from the code provided by Dave96 here:

How to draw conic gradient?

Here you go. It’s not the fastest implementation as it uses an intermediate image and fills every pixel but should work ok: //============================================================================== /** Fills and angle gradient. The gradient will start at -Pi radians (or 90 degrees to the left) and continue all the way round. */ inline void fillAngleGradient (Graphics& g, const ColourGradient& cg, Rectangle<int> bounds) { Image img (Image::ARGB, bounds.getWidth(), bounds.getHeight…

[fuo]

One thing I forgot to mention: some of the round icons in the projucer (compiled from the Direct2D source) will sometime disappear. I don’t know what triggers this, but it seems to happen when the projucer has been left open for a long time in the background, and possibly after a few windows sleep cycles.
I don’t know if this is an issue with the code or with my system (14900k with no dedicated GPU)

[matt]

Sorry for the confusion; please try the direct2d branch again.

Matt

[fuo]

Hi @matt

Thanks for the update!
The new version does matches the software and openGL very well in my plugin.
The main remaining difference is with font rendering.

I managed to track the issue I had with black images.
It looks like the BitmpaData object has to go out of scope for the image to be written to.

This is not the case with the main branch, but is the case with all renderers (software, openGL, Direct2D) in the Direct2D branch.

Here is a little example:

	void paint(juce::Graphics& g) override {

		bool drawWithinBitmapDataScope = true;

		juce::Image img{ juce::Image::RGB, getWidth(), getHeight(), false };

		{
			juce::Image::BitmapData data{ img, juce::Image::BitmapData::readWrite };

			for (int x = 0; x < getWidth(); ++x) {
				for (int y = 0; y < getHeight(); ++y) {
					data.setPixelColour(x, y, x == y ? juce::Colours::blue : juce::Colours::yellow);
				}
			}
		
			// draws black image:
			if (drawWithinBitmapDataScope) g.drawImage(img, getLocalBounds().toFloat());
		}

		// draws the actual image:
		if (!drawWithinBitmapDataScope) g.drawImage(img, getLocalBounds().toFloat());
	}

If drawWithinBitmapDataScope is true the image is black

It is probably an easy fix on your side, but it had me scratching my head quite a bit before I managed to reproduce the issue

I will now try to focus on the setBufferedToImage + scale situation and report back.

[matt]

@fuo

Regarding getPhysicalPixelScaleFactor, my use case is simple: I need to generate complex images and cache them, and keep them till the scale changes. The image also requires the correct final scale, and this is all done in the paint method.

OK, I understand. There are still some unresolved design questions about how to handle DPI scaling for images in general. I’ll have to get back to you.

Matt

[matt]

@fuo

I managed to track the issue I had with black images.
It looks like the BitmapData object has to go out of scope for the image to be written to.

In this case, I’m going to have to give you the old “it’s not a bug, it’s a feature”. At the risk of repeating myself, this is a consequence of storing the pixel data in the GPU. Creating the BitmapData object temporarily maps from the GPU to the CPU. When the BitmapData object goes out of scope, the image is unmapped and copied back to the GPU.

I agree that this behavior is different from previous versions of JUCE, but in this case I think that the performance boost for GPU-resident Images is so significant that difference is worthwhile.

I’ll see about adding an assert to check if a temporarily mapped image is passed to drawImage.

For your specific use-case, try explicitly creating your Image as a software image:

juce::Image img{ juce::Image::RGB, getWidth(), getHeight(), false, SoftwareImageType{} };

That image will be stored in CPU RAM, not GPU VRAM, and you should be able to work with it just as before.

Note that using a Graphics object to paint to an Image works similarly. If you haven’t, I recommend you review the wiki page about Direct2D Images:
https://github.com/mattgonzalez/JUCEDirect2DTest/wiki/2)-Images-Images)

I really appreciate your continued efforts.

Matt

[fuo]

I see, looks like I did not read the manual
Sorry about that, that makes sense.

I tried to track down the problem I have with setBufferedToImage, with components popping in and out of view, but I could not find the time to isolate it properly and reproduce it in a demo code.
I have bufferedToImage components within others, as well as buffered shadows, etc.
I also see shadows getting repainted over and over themselves (getting less and less transparent to the point of becoming a black shape) when the buffered object is repainted.
Once again difficult to isolate, but it is there.

One thing is easy to reproduce though: buffered components are not repainted when the AffineTransfrom scale changes, contrary to software renderer. This can easily be demontrated by setting setBufferedToImage for TestComponent in my demo code, and scalling up the window: the text becomes blurry because the image is not invalidated.

[fuo]

I think I found something else related to scaling:

In the example code I provided replace TestComponent::paint with:

void paint(juce::Graphics& g) override {
	g.fillAll(juce::Colours::white);
	g.setColour(juce::Colours::black);
	g.fillRect(50, 50, 50, 50);
}

(and no setBufferedToImage of course)

When the display scale is 100% (non retina display) all renderers show visible 1px “bouncing” of the different edges when resizing/scaling the window. This is normal and expected.

Now when switching to a 200% display scale (typical retina display), the software and openGL renderers show much less of that effect as the pixels are now 1/2 the size (1 physical pixel), whereas the direct2D renderer still show the same amount of bouncing (ie 2 physical pixels here).

[fuo]

Another example of this phenomenon, perhaps more illustrative of the consequences it does have on a layout.

Set you display scale to 200% and replace TestComponent::paint with:

void paint(juce::Graphics& g) override {
	g.fillAll(juce::Colours::white);
	for (int i = 0; i < 5; ++i) {
		g.fillRect(50, 50 * i, 50, 48);
	}
}

When scaling down the window the separations between the black rectangles starts to disappear as soon as you resize (and scale) the window down, contrary to the software renderer.

[kunitoki]

It is very important in my opinion to have consistent rendering tests before rolling out more graphics features or changes. Regardless of the backend used, the graphics behaviour should be consistent and reproducible.

[matt]

Sure.

I was thinking about your post and thought this would be a good time to go back and revisit the Direct2D test repository. As I was going through the repo, I found a bug with image painting. So, yeah, point taken.

I also put together a renderer comparison app that generates random components and painting operations and shows the software renderer side by side with the Direct2D renderer along with the difference between the bitmaps.It’s not much to look at but I found it helpful:

Of course there’s plenty more that could be done, but I usually feel that way about every project.

Matt

[matt]

Hi @fuo-

Sorry - sounds like my previous post came out wrong. My apologies! Looking at my post again I could see it coming across as critical, but that wasn’t my intent.

I’ll take a look at the other issues you posted.

Thanks again for all your detective work!

Matt

[fuo]

Hi @matt,

No worries at all, that was an attempt of humor on my part!
Let me also make it clear that I really appreciate all your work with this renderer: this is quite a fantastic achievement.
I am eager to use it in production, and it is probably better to tackle any potential issue now rather than when Juce 8 is out.

In that spirit, here is one more : in standalone mode, when the window is moved (or has been moved and is still grabbed) the content is repainted continuously.

[kunitoki]

Sounds great progress, now, this should be part of juce CI.

[remaincalm]

i’ve just pulled the most recent code and i’ve noticed a bunch of stuff broken now around drawImage from a bitmap to a different bitmap

i might just be holding something wrong and i’ll start running through the docs again to make sure i’m not doing anything goofy, but were there any intended behavioural/semantic changes that landed in the last two weeks?

thanks.

[matt]

I’ll take a look. Could you please give me a more specific example?

Alternatively, could you try this example and see if it works for you?

mattgonzalez/JUCEDirect2DTest/blob/pips/PIPs/Direct2D Image Edit Test.h

Matt

[remaincalm]

i’m getting black images on copy, and alignment seems wrong
(edit: see next reply for a possible culprit)

[remaincalm]

ok, works fine in
mar 12 6d6c192f3cb8da5e353688850925a0aa5641770a

first broken in the following commit, there may be other breakages afterwards:
mar 12 3720fa74d74c818b2ba64d6b796b053a507a3703

[Album] imgur.com

anything i should be watching for? i load a grid of knob graphics from a PNG loaded from a zip stream and use drawImage to copy it in an overloaded paint function, nothing too weird