Just wanted to confirm here that Matt’s image-permanence branch, as well as the mescal module he’s been working on, yields huge benefits with the D2D renderer. In particular, using the Image::disposable flag within JUCE library code has been invaluable—without it, we were seeing calls to initializeBitmapData that would hang for up to 10 milliseconds per frame due to blocking on a GPU memory mapping operation. Furthermore, the ability to request VRAM at startup seems to have a positive benefit on the performance of D2D, in particular with tessellation (which seems to be our biggest performance hindrance save for the bitmap hangs). Without these changes, the Direct2D renderer is more or less unusable for us without large revamps to our codebase, so I’m a proponent of getting those changes incorporated into JUCE itself as an alternative to a compile-time D2D disable flag.
Agree!!!
The plugins I make are all completely vector based / programmatically drawn (no bitmaps). Until recently, this was extremely problematic on the current Juce D2D renderer.
In addition to major slowdowns with any direct image methods (setPixelAt, multiplyAllAlphas, moveImageSection), and problems with drawing gradients efficiently, there was also a problem where multiple D2D rendered windows would lock up the windows message thread (and no mouse events would get processed).
With what I can only assume was a huge amount of work, all these issues have been fixed.
The result? All of my plugins are rendering (using D2D, on Matts image-permanence fork) at blistering speeds.
For example, here is a standalone synth which renders a 1000 point FFT, waveform (oscilloscope) and LFO waveform. Each is drawing at least 60 timed per second (you can see the FPS on the UI). This is on a 5 year old modest Windows computer with a pretty limited NVIDIA GeForce 1650.
I would really like to at least hear ANYTHING from the JUCE team about this. Locking up the message loop is a giant no no.
We had to offer an option to use the software renderer, so some customers could even use Nexus5 on their machine.
Now we have to look into cherry picking Matt’s branch into our branch because the official one is not fit for releasing any products with.
I am still on JUCE 7 but, I hope the difficulties with rendering will be fixed soon if possible, an simple switch to the old renderer would be very helpful.
In my plugins, there is a functionality where it requires an image to be fully rendered in the background thread, with multiple paint operations. Does it mean, with JUCE8, that at every paint operation the image is backed up into CPU-RAM? It is really important that this can happen at the usual speed. Is it possible to switch off this CPU buffering for an image object, so we still benefit from the Direct2D?
@chkn The problem you’re describing is precisely what Matt’s branch aims to fix. In its current state, JUCE 8 will always back up an image on the CPU. Matt’s notion of image “permanence” directly addresses this problem, but a solution to it does not exist in vanilla JUCE 8 as of now.
Hi @chkn-
If you’re painting to an image on a background thread then the issue you describe will be mitigated. I tried a quick test and you can still see good painting performance for your paint handler since the cost of the software backup is shifted to your thread.
So you’ll still see benefit from Direct2D even with the JUCE as is. Ultimately, we will need an official API for disabling the software backup for procedurally generated images.
Matt
Hi everyone-
Here’s a summary of the various performance improvements available here:
To take advantage of most of these improvements, you’ll need to mark your Image object as “disposable”:
auto image = juce::Image{ juce::Image::ARGB,
getWidth(),
getHeight(),
true,
juce::NativeImageType{},
juce::Image::disposable};
If you’re having performance issues with image painting, please try this branch and DM me if you have any questions.
Matt
Disposable Image support
Some of the Image methods were still only implemented as CPU software functions. These Image methods now run in the GPU as shader effects for disposable images.
Image::desaturate
Image::moveImageSection
Image::multiplyAllAlphas
Image::convertedToFormat
Component effects now use disposable images
Cached component images now use disposable images (setBufferedToImage)
Gradients
Cached gradients is now shared between device contexts- meaning you can reuse the same gradient object for painting different windows and images without the cost of recreating the gradient for each window or image. No code changes are necessary.
Painting
Removed the swap chain thread and async notifier to reduce message thread traffic
Limit painting to take no more than 80% of the message thread time
What is the reason for making it explicit and public to an image? It looks more like some cache mechanism detail that should be hidden within the ImagePixelData. There is a NamedValueSet for a reason. It also has no meaning in regards to SoftwareImagesType or OpenGL Images. So NativeImageType could just use another constructor, or even just a NativeDisposableImageType
and hide this whole mess. Component effects and Cached component could then use new NativeImageType.
To be honest I was really annoyed by the change from SoftwarePixelData to Direct2DPixelData. When I initialize an image for post processing, the default case should be a RAM backed storage. There is also the case where a big CPU atlas/sheet doesn’t need a GPU upload, because it selects a much smaller PixelRegion via Image:: getClippedImage.
It is only necessary to upload into a Direct2D image when it is really intended and queue to be rendered to a Direct2D context.
I advocate to avoid messing with the Image API too much and just improve the caching mechaism by auto-detecting frequently changing pixel content.
Please think about other use cases then your own when making these comments. Being able to say “only create this image on the GPU and never copy it back to RAM for any reason” has its use cases and we need a mechanism to do this.
Adding other image types just multiplies the number of possible enums and makes working with these unnecessarily harder.
You would suddenly need native, native disposable, native permanent, etc.
Matt’s implementation is fine, there’s no need to complicate it any further.
APIs trying to decide what’s right for my product always comes with compromises, and are never a good solution. I want a good solution. I want more control, not less.
I see where you’re coming from.
Please bear in mind these changes aren’t by any means official; this branch is meant specifically for testing and verifying various performance improvements. Mainly, I was trying to make it easier for other developers so they could test with a minimum of changes to their code.
I expect this will look very different when it’s actually incorporated into JUCE.
The question of the default image format is tricky. I’ve gone back and forth on that myself and I’m not sure there’s a clear-cut right answer. I’ve spent a considerable amount of time conversing with other developers about JUCE rendering and almost every one has a different approach. In retrospect I would have preferred that JUCE had a runtime D2D on/off switch for both images & windowed rendering.
Ultimately, I’m a JUCE user, not a JUCE team member, so API changes are not up to me.
Matt
Question - based on your previous posts, you obviously have have a lot of experience with OpenGL & Vulkan.
For DX11 & DX12, you have to handle unexpected adapter removal. I’ve been playing with Metal and there doesn’t seem to be an equivalent.
How do you handle that in OpenGL and Vulkan? Say, for instance, you unplug an external GPU, or the device restarts.
Matt
We’re working on it:
Damn tough to read this after your initial response made me question my own sanity.
You should definitely stay on 7, unfortunately if you were on the subscription your JUCE 7 license was automatically invalidated so moving to 8 was a forced upgrade.
Nothing personal to the JUCE team because I know it’s tough, but this is easily the worst upgrade experience so far. Why are we not beta testing these things with real production grade applications?
I think it’s safe to say we’re all praying that the solution is to promptly fix the software renderer while we actually field test D2D. I know that ideally we would fix D2D to work for everyone, and I agree that should be the focus, but we need to get back to an acceptable baseline here. It’s tough to fix an airplane mid flight.
Some of the terms around tiers and contributors have changed, but a JUCE 8 license grants you the right to continue using JUCE 7 (and all previous versions too). There is nothing forcing you off JUCE 7.
Apologies at the top of the EULA it states:
A perpetual licence grants the Licensee a license for a specific major version of the Framework under the terms of this Agreement in perpetuity.
But then it later states:
Each Licence includes new Minor and Point versions for the specified Major version of the Framework, and all previous Major versions of the Framework.
Here’s some unofficial methods to globally disable Direct2D:
void LowLevelGraphicsContext::setDirect2DEnabled([[maybe_unused]] bool enabled);
bool LowLevelGraphicsContext::isDirect2DEnabled();
These are static methods. For example:
direct2DToggle.onClick = [this]()
{
LowLevelGraphicsContext::setDirect2DEnabled(direct2DToggle.getToggleState());
Note that this will only affect images or windows created after you change the setting.
This is primarily meant for testing and debugging and is not an official JUCE change.
Matt
Same. I’d go so far as to say I was shamed for not wanting to use this yet and I’ve stepped away from the forum for a while as a result as I don’t need to put up with that sort of thing.
I’d still really like to simply see a Projucer switch so we can adopt Direct2D at our own pace while working on a Juce 8 transition. We have switches for some pretty trivial things, but we’re reading about unofficial ways to do this with a major feature that still seems to be causing some people problems.
Yeah,sorry about that. We didn’t have any issues until public release. No beta tester, developer or sound designer had any problems. Performance was great,
But when you suddenly have thousands of users, things change.
To be fair, it seems to affect only a small portion of our customers, but it’s still annoying and frustrating.
It’s not about my or your use case but about cluttering the public API. Keeping it clean and easy is in the common interest of all JUCE users. Image is a shallow holder of ImagePixelData and not responsible for any implementation details. An attribute like disposable is not common to all image types. Therefore ImagePixelData should handle all of this by getting it via something like NativeImageType(NativeImageType::Disposable). I mean do what you want but it is the way the Image API was designed and staying close to the intended usage would increase the probability of it finding its way into the main JUCE repository.
The thing that stands to debate is not the use-case but if we really want to make it explicit or instead just detect it in the PixelData implementation. No public Disposable needed.
Shouldn’t all use-cases of an image be detectable by the various access methods? If Image::BitmapData with Read mode is never called, we don’t need an implicit sync between the software pixeldata and direct2d object. If there is no software image source, we don’t need a readback either, cause we will never reupload it and instead just redraw the cached component image on context recreate.
All of this thinking about image internals and lifetime will ruin the simplicity. It’s on the JUCE implementers to keep this hidden but still solve the performance issues.
I think Vulkan handles device loss just as an error, see VK_ERROR_DEVICE_LOST. On any graphics queue related functions, this error would be returned, so it’s your responsibility to not use any remaining objects if you lose the device/context.
In the JUCE OpenGL implementation it would call context closing, removing any cached context objects, or probably GL_OUT_OF_MEMORY error if severe.
On Metal it seems to be detectable errors of command buffers
My take on this. Use the storage type of the source! If we load a PNG from disk, there is a higher chance we use it as SoftwareImage first and then at a later point draw it with Direct2D or OpenGL. Maybe we won’t. So we could delay the GPU upload until then. Unless I proactively load it into a Direct2d backed image.
If we use a temporary cached component image, we absolutely will most probably want a NativeImageType.
This is an imaginary case. But let’s say we had a CompressedImageType. Using something like BC7 texture compression for example. We would load it from disk and then at some point of actual use, there would be a transform into another ImageType, like OpenGLImageType(), or VulkanImageType(). Which would be device local GPU textures and read-only in the most common case, because one rarely writes to compressed textures.
Same for a (some day in the future) FloatTextureType. HDR displays and all that. We have a 32-bit float RGBA (.hdr, .exr) and load it from disk. Then at a later point decide if we want it in a RGBA 16f, or RGBA 32f, or even just in RGBA 8 texture. This shouldn’t be decided for us, unless we explicitly demand it.
Haha, I also switched because of your positive feedback 
Anyway, we are locked with JUCE 8 now. I hope those problems can be fixed as fast as possible. Our products are out in the field.
Please JUCE team, add feature toggles for changes like this and stop making these huge releases. Fast feedback is the way to go. Most of us don’t have the time to test feature branches.
Integrate features when they are ready with the possibility to disable them as long as they have open bugs. Nobody buys JUCE because of the features in the new release. It has some kind of subscription license anyway.
Edit: We have now also received the first complaint about the graphics performance