Direct2D

Windows
1

Hi everyone-

I’ve been looking into using Direct2D for our next JUCE application. I’ve been able to use a HWNDComponent and render Direct2D within that child window, so that’s one approach.

However, I’d prefer to use the standard JUCE graphics API, so I thought I’d take a deeper look at the state of Direct2DLowLevelGraphicsContext and try and get a handle on what’s missing.

The JUCE Direct2D renderer works well and performs well for the most part. The text rendering looks really nice. However, there are still some problems. Here are the issues I’ve run into:

  • excludeClipRectangle is not implemented

  • beginTransparencyLayer and endTransparencyLayer not implemented. This is easy to check by just enabling the JUCE splash screen.

  • Drop shadows may not render correctly, which may be related to the transparency layer stuff.

  • AttributedStrings don’t always draw correctly. This is easy to reproduce with the FontsDemo in the JUCE demo. Change the JUCE demo to Direct2D mode; the list of fonts on the left hand side of the window will be blank.

  • Judging by the comments in the code, there may also be performance issues with SavedState and a few other minor issues

I’m going to keep testing and see if I can patch some of this. Please let me know if there are any other outstanding issues.
All the best-

Matt

2

juce_win32_Direct2DGraphicsContext, line 212 is currently:

void Direct2DLowLevelGraphicsContext::SavedState::clipToPath (ID2D1Geometry* geometry)

But that won’t build; instead it should be just this:

void clipToPath (ID2D1Geometry* geometry)
3

Looks like the font names aren’t drawing properly in the fonts demo because DirectWriteTypeLayout::setTextFormatProperties currently only handles horizontal text alignment, not vertical alignment.

I changed setTextFormatProperties to call IDWriteTextFormat::SetParagraphAlignment and now the fonts demo looks right.

4

Direct2DLowLevelGraphicsContext::setInterpolationQuality is not implemented.

5

Implementing beginTransparencyLayer and endTransparencyLayer turned out to be straightfoward:

void Direct2DLowLevelGraphicsContext::beginTransparencyLayer(float opacity)
{
currentState->beginTransparency(opacity);
}

void Direct2DLowLevelGraphicsContext::endTransparencyLayer()
{
currentState->endTransparency();
}

Then I added these to SavedState:

void beginTransparency(float opacity)
{
    auto hr = owner.pimpl->renderingTarget->CreateLayer(nullptr, transparencyLayer.resetAndGetPointerAddress());
    if (SUCCEEDED(hr))
    {
        owner.pimpl->renderingTarget->PushLayer(D2D1::LayerParameters(D2D1::InfiniteRect(),
            nullptr,
            D2D1_ANTIALIAS_MODE_PER_PRIMITIVE,
            D2D1::IdentityMatrix(),
            opacity,
            nullptr,
            D2D1_LAYER_OPTIONS_NONE),
            transparencyLayer);
    }
}

void endTransparency()
{
    if (transparencyLayer)
    {
        owner.pimpl->renderingTarget->PopLayer();
        transparencyLayer = nullptr;
    }
}
6

Very cool that you’re doing this. Implementing the interpolation quality is also relatively easy. Theres a day fferent quality enum since windows 8, so there has to be a runtime check on the windows version.

I noticed, that direct write fonts currently can’t be loaded from memory - this caused a crash for me in the direct2d renderer.

7

Thanks; yeah, looks like Windows 8 also has better gradient filling using ID2D1DeviceContext::CreateGradientStopCollection.

Do you have a short, reproducible example of the problem you mentioned with loading DirectWrite fonts from memory?

Matt

8

The drop shadow problems are due to an extra unwanted affine transform; when Direct2DLowLevelGraphicsContext::SavedState::createBrush makes the gradient brush, it should just be applying the transform from the fill type. So instead of this:

D2D1_BRUSH_PROPERTIES brushProps = { fillType.getOpacity(), transformToMatrix (fillType.transform.followedBy (transform)) };

it should be this:

D2D1_BRUSH_PROPERTIES brushProps = { fillType.getOpacity(), transformToMatrix (fillType.transform) };

Otherwise the brush is transformed again in Direct2DLowLevelGraphicsContext::fillRect.

This is fun. I’m going to bed.

Matt

9

I made a fork:

3 Likes
10

HWNDComponentPeer needs to call Direct2DLowLevelGraphicsContext::resized when the window changes size.

Comments, critiques, and suggestions are most welcome.

11

I just saw this, and I feel like I have to comment:

About a year ago, we did exactly what you’re doing right now. We made it fully work, with memory-based fonts and everything. First, we were very excited, probably similar to how you feel right now. Only after the majority of the work was finished, the disappointment set in.

The good: it’s about twice as fast as using OpenGL to render the standard JUCE components.

The bad: since JUCE splits all paths (thus almost everything) into separate scan lines and then splits those into sometimes individual rectangles, the rendering was STILL slower than the software renderer (overall, combining both threads).

The OpenGL and Direct2D low-level contexts feed their drawing commands to a background thread executing the actual drawing. If you measure the CPU time spent in your main thread AND the drawing background thread, you will realize that both together still take more CPU than just using the software rasterizer.

Of course, if you do simple tests and draw a giant rectangle and nothing else on the screen, then both the OpenGL and Direct2D contexts will be a lot faster. But as soon as you add some real components, texts, gradients, etc., it all comes crumbling down.

On top of that, the Direct2D gradients use a different gamma correction than OpenGL and the software renderer, and we couldn’t find a way to change/disable it. So your gradients will look notably different.

The text will look slightly different too. If I remember correctly, it looked quite a bit thinner. Maybe the gamma correction again?

Even if you think you can live with all that, both the OpenGL version and the Direct2D version seem to work just fine, but when you open multiple instances of your plugin, then the fight over the resources starts. We had three windows open and suddenly the background thread was overloaded with drawing commands, slowing the whole thing down to a crawl. It was a stutter-fest. All three windows stuttered like crazy. Way worse using the software renderer. Forget about opening a fourth or even fifth window.

This was on an i9-9900K / GTX 1060, build in release mode. Written by a 20+ year veteran from the video games industry.

The way JUCE builds the rendering paths, breaks them into scan lines, etc. is simply not workable for an HW accelerated UI.

Sorry to be the bearer of bad news and good luck.

6 Likes
12

Thanks for sharing your experience; I remember you posting previously that you had also worked on the Direct2D renderer and found it unsatisfactory.

For my purposes, I’m hopeful the Direct2D renderer will work well. I prefer how the text looks in Direct2D mode, and we’re just doing applications, not plugins, so hopefully we won’t run into the resource contention issues you mentioned. I am concerned about how much the saved state context creates D2D objects on the fly; as one of the comments mentions, I think it would be preferable to recycle existing objects. Plenty of room for improvement there.

Did you ever try implementing Direct2D 1.1 or 1.2? Looks like the current JUCE code is based on the first version of Direct2D.

So far this is mostly something I’m doing for fun to teach myself something new. It may well turn out to be disappointing!

Matt

13

Just fixed Direct2DLowLevelGraphicsContext::clipToRectangleList and clipToPath not applying the current transform.

14

The other approach I’ve been playing with is bypassing the JUCE Graphics layer by calling Direct2D from my application using a JUCE HWNDComponent. Of course this isn’t cross-platform, but it does cut out the middleman (so to speak).

Matt

1 Like
15

Can you think of an elegant way to pass the original paths down to the low level graphics context? I’ve implemented a renderer on top of skia a while ago that also appears to suffer from this conversation to scanlinines.

16

My fix for vertical justification for text layouts broke the demo app, so I reverted my changes to setTextFormatProperties and came up with a semi-cheesy manual solution.

I implemented excludeClipRectangle. That one was tricky; as far as I can tell, Direct2D doesn’t have a parallel method for excluding regions. I ended up using an opacity mask bitmap (very similar to clipToImageAlpha); I’m not sure if that’s the best solution, but it seems to work.

I also changed getClipBounds to handle clipping to a RectangleList. I don’t have a good solution for getClipBounds handling clipping to an image or a path, but I’m not really clear on the rules for how those should work. I did some experiments with the software renderer and that didn’t always seem to reduce the clipping region as I expected, so perhaps I’m missing something.

Matt

17

I was able to reproduce the crash when using a custom typeface. I’ll check it out.

Matt

18

Loading fonts from memory with Direct2D is nontrival, but entirely possible. Looks like there are two different approaches; making a custom font collection or a custom font set. Custom font sets are only supported on Windows 10; custom font collections work with Windows 7 or later.

That, in turn, brings up the whole question of what version of Windows this should target. The original JUCE Direct2D renderer is at least ten years old, which would have been Direct2D 1.0. Subsequent versions of Direct2D have added some nice features (more efficient rendering, better geometry support, better gradients, etc). So does anyone still care about supporting Windows 7 and 8? Or would it be better to just focus on Windows 10 / 11?

Matt

19

Hi everyone-

I think I’ve fixed all the major issues I know about and need to set this aside for a bit. The only major problem I’m aware of is loading custom fonts from memory.

I wrote a stress test app to render lots of paths and text with multiple simultaneous windows and so far I’m encouraged. The Visual Studio profiler shows that most of the CPU time is in the JUCE glyph code, but I think I’m just scratching the surface in terms of profiling and performance tuning.

So it’s functional, but I’m sure there’s lots of room for improvement. It’s all up on Github if anyone wants to look at it. Comments and suggestions are welcome.

If anyone is interested in rendering Direct2D directly (no pun intended) inside an HWNDComponent and bypassing the Graphics layer entirely, let me know and I’ll post that as well.

Matt

3 Likes
20

Hi everyone-

I implemented a DirectWrite custom font collection loader which allows custom TTF fonts to be loaded from memory in Direct2D mode.

image

image

It’s all up on Github. Still needs to be reviewed and commented.

This implementation is a little clumsy since I am creating a custom font collection for each custom font; it would be nice to just have a single custom font collection to hold all the custom fonts.

There are still a few places where the JUCE demo app doesn’t paint properly in Direct2D mode; I’ll sort those out next.

Matt

3 Likes