Direct2D

Windows
21

Turns out the clipping needed a lot of attention, especially using transformed components (e.g. ComponentTransformsDemo). Direct2D does support rectangular clipping, but that clips differently when transformed (see PushAxisAlignedClip), so that won’t work.

I rewrote all the clipping to use geometry layers, which cleaned the code up nicely. I’ll need to test the performance more thoroughly, but so far it works fine.

I found a bug in the path rendering; every call to ID2D1GeometrySink::BeginFigure needs to be matched with a call to EndFigure, but sometimes JUCE paths can have a startNewSubPath marker without a matching closePath (see the toolbar tab in the WidgetsDemo). That just needed a little extra logic.

restoreState wasn’t restoring the last brush color; that’s fixed. There are probably other issues with popping the Graphics stack and restoring the previous state properly.

All pushed to Github.

Matt

3 Likes
22

I turned on the Direct2D debug messages which identified some object leaks and a performance improvement with clip layers. All sorted.

Matt

2 Likes
23

I’ve made a few more fixes to clean up the custom font collection loaders, and fixed a minor issue where painting an AttributedString would clip the bottom of the font descenders.

I think this is now at the point that I can start using it for my apps. If anyone else would like to try it, bug reports and performance reports would be most welcome. Note that this will only work on Windows 8 or later.

The next phase is to switch to ID2D1DeviceContext to take advantage of D2D1_DEVICE_CONTEXT_OPTIONS_ENABLE_MULTITHREADED_OPTIMIZATIONS and better path rendering.

The main performance issue is that the Direct2D renderer has to create a lot of temporary geometry and brush objects and then discard them. We could get better performance with Direct2D if JUCE graphics objects had a platform-specific peer, similar to a JUCE ComponentPeer. For example, if a JUCE Path also had a PathPeer, then each component class could retain the PathPeer as a member and not have to recreate it for every paint call. But I don’t know how that might affect other renderers, such as Vulkan or OpenGL.

Anyhow, I hope this proves useful to someone. Enjoy!

Matt

2 Likes
24

I’m no expert in Direct2D whatsoever, but for the Path → PathPeer matter, perhaps you can

  1. inherit from Path a derived class named PathWithPeer that holds a platform-agnostic pointer to such peer
  2. Inside your Direct2D code, when you deal with a Path, if you can dynamic_cast it to a PathPeer, you can store the peer there and reuse it in later renders
  3. if this proves to really improve rendering performance, that will represent a strong argument in favour of moving the “peer” member up in the Path base class in JUCE code
25

Sure, that could work. I was also considering using a dynamic cast with Graphics::getInternalContext to get direct (no pun intended) access to the Direct2D renderer.

To your point #3, I think for now I’ll focus on benchmarking to see how much a difference all this really makes.

Matt

26

I’ve updated the renderer to support Direct2D 1.1 and JUCE 6.1.

Direct2D 1.1 requires Windows 8 or later. This change should open up access to some of the newer Direct2D features like faster path rendering and changing the image resampling quality.

Matt

3 Likes
27

I’ve been lurking this thread long time ago, and I like to test it in my real-world app with hundreds of components and paths. However since I have Juce hacked (to allow things like capturing right key modifiers), I’m not sure how to test it. Should it be enough with downloading your Direct2DLowLevelGraphicsContext? or did you modify another files?

1 Like
28

Hi @juan1979 - thanks for offering to check this out.

Since you have your own modified version of JUCE, I recommend you update your fork to JUCE 6.1.2 and then merge my direct2d branch into your fork.

https://github.com/mattgonzalez/JUCE/tree/direct2d

I had to change several files:

modules/juce_graphics/juce_graphics.cpp
modules/juce_graphics/native/juce_win32_Direct2DGraphicsContext.cpp
modules/juce_graphics/native/juce_win32_Direct2DGraphicsContext.h
modules/juce_graphics/native/juce_win32_DirectWriteTypeface.cpp
modules/juce_graphics/native/juce_win32_DirectWriteTypeLayout.cpp
modules/juce_gui_basics/native/juce_win32_Windowing.cpp

and I added one new file:

modules/juce_graphics/native/juce_win32_DirectWriteCustomFontCollection.cpp

Once you’re finished merging, you’ll need to set JUCE_DIRECT2D=1 in your preprocessor definitions.

Direct2D is not enabled by default, so add this to your main window constructor:

getPeer()->setCurrentRenderingEngine(1);

Alternatively, you could add a renderer select control, similar to the JUCE DemoRunner.

Enjoy!

Matt

29

Great, I will test this weekend and I will report you back, thanks!

30

There are still some bugs lurking:

  • The clipping region reported by Graphics::getClipBounds doesn’t always match between the software renderer and Direct2D mode.

  • Sometimes if the app window is occluded and then brought to the front, the child components don’t paint.

There are probably other unresolved issues; please consider this to be a beta, at best. There’s still lots of benchmarking to be done.

Matt

31

Be sure to grab the latest commit; turns out the Direct2D 1.1 swap chain I’m implementing always needs to render the entire window. I think I can improve on that using a different swap mode, but this should at least keep entire components from vanishing.

Matt

32

Hi everyone-

I’m finally getting back to this. I’ve been trying to benchmark the Direct2D renderer versus the JUCE software renderer to see if Direct2D is actually an improvement. My answer is a qualified “yes” - the D2D renderer is much better at doing things like rendering paths with thousands of points or other CPU-intensive operations. However, there are still a few outstanding issues that need to be solved:

  • Smooth animation with multiple windows and a single process
  • Window resizing
  • Flicker

That’s the executive summary. Buckle up if you want more detail.

Stress Test App

I built a renderer stress test app that spawns multiple windows and attempts to animate all of them at once.

D2D stress test
D2D stress test3440×1440 168 KB

It’s probably hard to make out details in the screen shot; the top bar is a window with controls to select the renderer type, timer rate, etc. Most of the lower area of the screen is divided up into separate windows; you can sort of see the red outlines of each window. This particular setup is trying to run scroll the rectangles horizontally at 60 Hz.

The app can select from multiple renderers:

  • JUCE software renderer
  • JUCE Direct2D 1.1 renderer
  • Direct2D 1.0 (not using the JUCE renderer, just native Direct2D)
  • Direct2D 1.1 (not using the JUCE renderer, just native Direct2D)

This particular PC has an Intel Core i9-10900 CPU, 32 GB of RAM, and an NVIDIA GeFore RTX 2080 Ti.

The JUCE Direct2D renderer so far seems to perform similarly to the native D2D code, which is encouraging. But…

Direct2D with multiple windows from a single process

Animating multiple windows simultaneously from a single process doesn’t work very well; the animation hitches and jerks. Weirdly, if I spawn a multiple child processes, the animation is butter-smooth. Looks great! But that’s probably not a practical solution.

So that’s just confusing! Clearly the hardware can keep up. It suggests that there is some internal synchronization within Direct2D that is handled differently for single processes versus multiple processes.

This is using Direct2D 1.1, which is really Direct3D. Apparently Direct3D has a similar issue.

So…

Flip-model swap chain

Direct2D 1.1 uses DXGI swap chains. DXGI swap chains support a number of different swap modes.
The multiple-window single-process test was using the “blt” present model, which is the older swap mode. Microsoft really wants everyone to switch to the new flip model:

https://docs.microsoft.com/en-us/windows/win32/direct3ddxgi/for-best-performance–use-dxgi-flip-model

So I tried the flip model, and multi-window-single-process animation looks great. So, problem solved!

Except…

Window resizing

Resizing a window with the flip model swap mode is rough. Dragging the mouse results in unpleasant graphical artifacts around the window edges. Window resizing with the blt mode is perfect. Searching around shows that this is a fairly common issue.

Also…

Flickering

I tried the flip model swap mode with one of my apps and saw a lot of visible flicker drawing the UI. Nasty. Again, using the older blt mode is fine.

Next steps?

Using the flip model really would be preferable. So the challenge seems to be solving the window resizing and flickering. Again, I’d be happy to provide even more detail and documentation if anyone else is interested.

Take care-

Matt

2 Likes
33

I dig it. Super cool that you’re keeping up with this!

Bearing in mind this is off the top of my head from ~5 years ago so could be wrong here - here are some suggestions.

From what I recall, D2D performs much better when there’s 1 DC per window. Is that what you have set up? I understand that that would make it challenging to use resources across windows because you would need to copy them to make them work.

If you happen to be doing what I suggested above, you shouldn’t have this issue.

The other alternative is to not try to dynamically resize - as games do it. This doesn’t look great for apps but would reduce the amount of rect invalidation (and resource recreation if need be).

Apparently someone has found a hybrid solution as well: https://www.gamedev.net/forums/topic/696236-trying-to-figure-out-smooth-window-resize-with-direct2d/5376943/ It could be worth checking out.

34

There’s a flag D2D1_DEVICE_CONTEXT_OPTIONS_ENABLE_MULTITHREADED_OPTIMIZATIONS that might help

35

Each window has one ID2D1DeviceContext, along with one IDXGISwapChain1 and one ID2D1Bitmap1. I am using a single Direct2D factory to create these; I’ve tried using a unique factory for each window, but I don’t think it made any difference.

Thanks for the link to the hybrid solution; I attempted a similar approach unsuccessfully, but it’s worth revisiting.

I also found this posting that suggested that using WS_EX_NOREDIRECTIONBITMAP for the extended window style solves the resizing problem:

https://stackoverflow.com/questions/63096226/directx-resize-shows-win32-background-at-edges

but that flag just made my window vanish entirely. Admittedly it no longer glitched while resizing…

Matt

36

Hi Jeff-

Thanks for the suggestion to try the multithreaded optimization flag; I’ll give it a shot.

Matt

37

I tried this in my (non JUCE app) unfortunately, it made no difference. I still get the window frame glitching horribly when resizing. (there’s meant to be scroll bars between the frame and the content)
image

38

Have you tried synchronising the drawing work with the display (if that’s not already handled automatically by the swap chain)? Something I’ve experimented with before is having a dedicated thread spinning on

and then calling a set of listeners to do the actual drawing of any batched dirty regions. This will likely require changes in the Peer, rather than the D2D renderer, but it might improve things.

Adding similar functionality, where JUCE could provide a callback to a Peer at the screen refresh rate, is something we would like to investigate more throughly. Some very preliminary testing has shown that a similar approach could be used to improve the smoothness of animations on macOS.

39

Microsoft themselves can’t fix this resizing issue…

I’m giving up for now.

40

Hi Tom-

Supposedly IDXGISwapChain1::Present synchronizes with the vertical blank:

https://docs.microsoft.com/en-us/windows/win32/api/dxgi/nf-dxgi-idxgiswapchain-present

I typically specify a value of 1 for SyncInterval.

As far as the peer, I’ve had to make a few minor changes to juce_win32_Windowing.cpp, but nothing too invasive.

Matt