Text rendering suggestion: Need Jules' eyes

General JUCE discussion
1

(Don't worry Jules; this isn't about font hinting!)

I've messed around with Juce's text rendering a lot, because I wasn't really satisfied with it (on Windows). It looks great on OSX, because the default renderer uses OSX's subpixel renderer - however, on Windows it uses a software antialiased approach. The problem is, that the gamma-change you introduced, while generally improving appearance, made it seem like different characters had different weights.

I then decided to create a subpixel renderer for fonts, that overrides the LowLevelSoftwareRenderer's antialiased text renderer, and so far, I'm pretty happy with the results! As is common with subpixel text rendering, the rendering inherits a slight, fussy feel, but will on the other hand vastly improve italics, kerning & spacing (it does subpixel positioning, also) as well as the gamma weight problem (it creates much more consistent intensity).

-- some examples --

edit 12/02, current progress and comparison of most used platforms (the render in this topic is called 'cpl render'):

860×66

Open it here for full resolution: http://i.imgur.com/FnFVED1.gif

 

 

Here's some text that illustrates the weight problem. Unfortunately, I didn't think of tagging the images with what renderer is used (the smoother one), but I hope it is obvious (otherwise, zoom in a bit to see the subpixel fringing).

This one will be resized in here, I think, so I'll link it instead (shows italics and a lot of different fonts):

http://i.imgur.com/Wump5Di.gif

There are some weird dots and pixels around in the images because of the compression in the GIFs.

Now, the implementation.. And the reason for this topic. This renderer only requires two minor changes to the repository, I obviously hope you will implement!

The code is basically an extended version of the EdgeTable::EdgeTableIterationCallback, which means it uses the exising font+glyph cache and edgetables! What it does is to intercept LowLevelGraphicsContext::drawGlyph, grabs the current font and scales it horizontally by 3 (to achieve the subpixel resolution).

This was my version of the intercept: http://pastebin.com/A6uhaKuK

And, here's the actual renderer (should you be interested?):  http://pastebin.com/SP2PW6rr
 

Here's a link to the latest version: https://dl.dropboxusercontent.com/u/41702019/CPL.SubpixelFonts.zip

What needs to be changed?

using namespace juce::RenderingHelpers;

​typedef CachedGlyphEdgeTable<SoftwareRendererSavedState> GlyphType;

typedef GlyphCache<GlyphType, SoftwareRendererSavedState> GlyphCacheType;

GlyphCacheType& cache = GlyphCacheType::getInstance();

// get the glyph from the cache

if (juce::ReferenceCountedObjectPtr<GlyphType> glyphRef = cache.findOrCreateGlyph(font, glyphNumber))

cache.findOrCreateGlyph() is private, and thus either needs to befriend LowLevelGraphicsContext or be public.

CachedGlyphEdgeTable::edgeTable is also private. I suggest creating an accessor method:

const juce::EdgeTable & CachedGlyphEdgeTable::getEdgeTable() const;

 

And that is all! Then it should work. Regarding performance, it is about 20% slower rendering bulks of text - but that really isn't much, considering subpixel rendering usually takes a lot more memory and performance! On the other hand, it doesn't actually create a copy of the glyphs edgetable on the heap, it uses the existing one. This also means it can't use the renderer stack to clip it. This may be a bad decision, I'm not sure (currently, it checks clipping inside the renderer loop).

If you're interested in the code, you can have it! I can create a version which is void of my math library, otherwise, I'm very interested in what you think about both the font rendering, design and / or proposed changes to the library. I know you generally want to use the platform to render text, but I'm getting impatient on Windows. :)

Regards

 

2

Cool! I came hopelessly unstuck when I had an attempt at this a couple of years ago :)

3

Does you code make the assumption that the display has the standard RGB arrangement, this could be a problem if the physical display is different/scaled etc.

The only real way to do this to use the original subpixel rendering methods from Windows, isn't that possible?

4

But i agree the current text rendering on juce windows isn't satisfying, especially with small fonts, 

5

And DirectDraw isn't better than the software font rendering ?

6

That does produce a nice result!

Unfortunately your implementation doesn't help me too much, as the way I'd implement this would be to modify the normal renderer, and get sub-pixel rendering for shapes as well as text. It'd also need to check the monitor to get the RGB order, and disable itself on higher-res displays, etc.

(I'm happy to make the tweaks you requested though)

7

@Andrew 

Thanks. Yeah same, had a breakthrough in the middle of the night though, after I told myself I was done with this. 

@chkn

Yes it does (it can support any though). Consider it a beta for now. As far as I know, there's no way to obtain the subpixel matrix order - beyond maybe query the order the system uses (which may be wrong as well) - if anyone knows how do this on Windows, please chip in. In case of Windows, the user must manually tune ClearType to support BGR displays.

One can check if the physical display is scaled using getPhysicalScaleFactor(), right? And it's already disabled for transformed fonts (it could actually work, I just haven't looked into it..)

@otristan

It may or may not. I'm not actually certain how it works, but assuming cleartype doesn't hint the fonts, it should produce the same output (depending on the gamma). The software renderer already uses directwrite to extract the glyph outlines.

In any case, I chose this approach because it works out of the box and doesn't major require modifications to the repository, while still supporting everything in the Font/Typeface classes. It even works if you're using FreeType to generate hinted glyphs.

@Jules

Thanks! And thanks for the repository changes. And yes, the RGB order. I'm not so sure of that aswell. However, as long as you adhere to left-to-right horizontal scanline rendering, the barebones of this code can render anything that's already anti-aliased. The problem is having to scale everything by 3.. ideally, much of the render code should just be done in floating point so you preserve subpixel offsets and such. 

I may look at it soon, and see how it could be done.

8

Looks great !

I wonder if it would be possible to use this to render text on Mac OS X as well. One of the issues I have JUCE right now is that the text rendering can look too much different on Windows and Mac OS X, and this may be a solution...

9

Thanks.

Yes it can, however this should already approximate the OSx rendering. I'll make a comparison between native and osx rendering later.

10

Hi Mayae, can you make a version without your Math library? I really would like to have this on windows and for fonts only - like on OSX where only fonts get subpixel rendering.

I'll try to figure out how to detect ClearType (&screen rotation) settings on windows in the meantime. I tried getting your code to work, but I don't understand what this line does - all the other Math lib stuff seems clear.

                                auto intensity = cpl::Math::roundedMul(gamma(alpha), colourSetup[3], weightLut5[i]);

11

this might be a big help for detecting ClearType settings. It's a description of how the cleartype calibration information is stored in the windows registry. Screen rotation would also need to be taken into consideration.

https://msdn.microsoft.com/en-us/library/aa970267.aspx

12

Here's the relevant portions of the library: http://pastebin.com/kmjgs9dM

It's just fixed point multiplication with added rounding.

13

ok. thanks.. trying it out now. Btw. I think there's a minor rounding bug in the roundedMul with three operands.. I think the constant add should be 0x8000 as it's shifted by 16 bits and not 8 like the two operands one.

 inline std::uint8_t roundedMul(std::uint8_t a, std::uint8_t b) noexcept
 {
    return static_cast<std::uint8_t>((static_cast<std::uint_fast16_t>(a)* b + 0x80) >> 8);
 }

inline std::uint8_t roundedMul(std::uint8_t a, std::uint8_t b, std::uint8_t c) noexcept

 {
       return static_cast<std::uint8_t>((static_cast<std::uint_fast32_t>(a)* b * c + 0x80) >> 16);

 }

14

That would make sense, yes.

15

The renderer is now running correctly for me. I will now try to write detection code for those registry values.

16

Good stuff. I'm unfortunately a bit busy the next couple of days, so can't really work on it now.

17

Here are some intermediate results: I figured the best way to detect ClearType settings is the windows call SystemParametersInfo. If can retrieve all the settings we need besides screen rotation. I looked into screen rotation as well and it for sure is tricky because there could be multiple displays with different orientations. Additionally the only windows desktop call to fetch rotation (EnumDisplaySettingsEx) appears to be buggy if multiple users are logged in!

I figured ClearType supports RGB and BGR subpixel layouts. I thought maybe it switches if the screen gets rotated. I was at least convinced it would render the other way for flipped landscape, but it turns out it does not! Basically cleartype cannot handle screen rotation correctly. That came as a big surprise. It just keeps its subpixel order no matter what. There used to be a 3rdparty tool called ClearType Rotator that updates cleartype settings if the screen rotation changes. 

Overall it seems to me the best way to go about this would be to just replicate what cleartype does (meaning just ignore rotation). If someone would use ClearType rotator we'd also get the changing settings.

In order to use detection I added a flag "useSubPixelGlyphs" to your graphics class. If false, it just calls normal rendering. I also needed that to avoid the throw for non-RGB pixelbuffers that occur on windows for Drag operations. Then I do this every time the graphics object is created. 


                            #if JUCE_WINDOWS
                                BOOL smoothing = false;
                                SystemParametersInfo(SPI_GETFONTSMOOTHING, 0, &smoothing, 0);
                                if (smoothing == false) {
                                    useSubPixelGlyphs = false;
                                } else {
                                    UINT smoothingType = 0;
                                    SystemParametersInfo(SPI_GETFONTSMOOTHINGTYPE, 0, &smoothingType, 0);
                                    if (smoothingType != FE_FONTSMOOTHINGCLEARTYPE) {
                                        useSubPixelGlyphs = false;
                                    } else {
                                        //UINT gamma = 1400; // 1000 - 2200
                                        //SystemParametersInfo(SPI_GETFONTSMOOTHINGCONTRAST, 0, &gamma, 0);
                                        
                                        UINT splayout = FE_FONTSMOOTHINGORIENTATIONRGB;
                                        SystemParametersInfo(SPI_GETFONTSMOOTHINGORIENTATION, 0, &splayout, 0);
                                        if (splayout == FE_FONTSMOOTHINGORIENTATIONBGR) {
                                        }
                                    }
                                }
                            #endif                
 
Now I just need to change your scanline renderer to support BGR pixel order and send an parameter to it. Maybe the gamma from cleartype should also be used, but I'm not sure how to do that without messing up the JUCE gamma.

I also tried your code on OSX and at least in my project it's not working as it appears OSX is always using 32-bit image buffers even if an image doesn't need an alpha channel. It would be good if the scanline renderer could render to that as well.

Lastly I ran into a problem with how JUCE creates graphics objects for images. On windows using the Graphics(Image&) call always produces a LowLevelGraphicsContext, so I didn't get subpixel fonts for rendering to images. I use a custom LookAndFeel to create the SubPixel-Graphics and so I ended up with this for images drawn inside a component:


#if JUCE_WINDOWS
            ScopedPointer<LowLevelGraphicsContext> context = getLookAndFeel().createGraphicsContext(image, Point<int>(0, 0), imageBounds);
            Graphics gi(*context);
#else
            Graphics gi(image);
#endif

 

this calls the override in my lookandfeel and leads to subpixel rendering (for RGB image buffers...) 


LowLevelGraphicsContext* MyLookAndFeel::createGraphicsContext (const Image& imageToRenderOn, const Point<int>& origin, const RectangleList<int>& initialClip)
{
    return new cpl::rendering::CSubpixelSoftwareGraphics(imageToRenderOn, origin, initialClip);
}

 

18

I agree with your thoughts on cleartype, and thanks for investigating it - i will add it to the OP soon. As for the gamma, I'm not sure what scale 1000-2200 represents, but if you can convert it to a fraction, you can just pass it as the last parameter to the renderer.

For BGR displays, it's very easy to get it working on Windows: Since the endianness of RGB (and ARGB) pixels are: BGRA, you can map the subpixeloffset directly to the pixelpointer (instead of the current modulo operation). On the other hand, on OSX you will then use the default Windows code. I would suggest making a template specialization so you dont have a runtime conditional.

 

I haven't tested it on OSX at all, because in my opinion the CoreGraphics does the job as it should (and it can take advantage of retina scalings better), so I haven't really bothered with it originally. It may be relevant when writing to Images, and yes, I've also wondered why the Graphics::createLowLevelContext() doesn't use the lookAndFeel method.

But yeah I just tried it and also wondered why Images are ARGB. Note that on Windows, Bitmaps are still 32-bit, HOWEVER the alpha channel isn't used and as of such Image::getFormat() returns RGB. This isn't the case on OSX, and I don't know how one is supposed to differentiate them. And in general you can't render subpixels onto ARGB images, because you then need alpha channels for each colour to do correct antialiasing.

 

Also I just noticed I haven't considered scaling factors like DPI and such.. Not sure how that will affect it.

19

You actually made sure it does not use subpixel rendering if there are scaling factors with the transform.isOnlyTranslated condition. That also means subpixel rendering is not used on OSX retina resolutions where the transform has a scale of 2.0f. I agree the whole thing is not really necessary on OSX because it does a very fine job, however as someone above wrote, it would make sense if one needs the look of fonts to match 100%. In my experience font weights look differently when comparing the juce renderer with the OSX renderer and there are small differences when it comes to hinting and spacing.

The gamma value IMHO is the standard monitor gamma scale. I'm not sure how to convert that to juce values. And what if other code already sets the JUCE gamma.. then we'd get a collision - but I'm not familiar with how gamma is handled.

20

True, there would be a point in making it work similarly on all platforms. I'll look at it soon.

There's no such thing as the JUCE gamma - JUCE doesn't account for this (normally the system does this at the hardware output level). The only place is the font antialiased font rendering, where all values are scaled by a hardcoded 1.6. So there would be no possible collision. For reference, here's how to actually do gamma correction:

So if you want to render a pixel with intensity 100, you'd raise it to the reciprocal power of the current gamma correction setting. If windows sets the gamma to 2200 (2.200), the gamma function would look like this:

// 0 >= x <= 1
std::uint8_t gamma_law(float x, float gammaSetting)
{
    return static_cast<std::uint8_t>(UCHAR_MAX * std::pow(x, 1.0f / gammaSetting) + 0.5f);
}

Obviously we can't have a floating point std::pow in the middle of each pixel render, so we'll have to figure something out - perhaps a linear interpolated static lookup-table? In any case, the gamma law should probably be a template functor to the SubpixelScanlineRenderer class.