In the meantime I managed to make it work with BGR and set up detection with some clumsy code I'd rather not show :). As I looked more at the subpixel renderer code, I'm wondering: why does it need 5-point lowpass filtering? I did some subpixel scaling of images in the past and all that was needed was starting with an image of 3x horizontal resolution and then shifting two colour channels by 1 pixel and then downscaling 3x horizontally again which just took the the linear average of each (shifted) 3-pixel group.
If I look at the render result of your renderer next to Cleartype, it does look horizontally blurred in comparison - would it make sense ot adjust that low pass filter? I of course already tried with just a 3-point filter and the result to my eye looks more like what cleartype does.
Reasoning for 5 point low-pass is presented here: https://www.grc.com/cttech.htm
Thanks for the link. Oh I've seen that before. After reading through it, I'm not convinced it's the perfect solution. The article suggests two steps of 1/1/1 horizontal smoothing to get rid of color banding, leading to the 1/2/3/2/1 filter kernel used. The article says one step is not good because the energy spreads out too much, so the second step is added to have a stronger center.
But why not just use a kernel with a stronger center in the first place? Imho binominal coefficients would be a better choice to use as they nicely sum up to powers of 2 - (no rounding problems with uchar coefficients) and have a stronger center. So I tried using a 1/2/1 kernel and a 1/4/6/4/1 kernel and I think both perform better than the ones in the article, less blurry but still nicely antialiased.
So basically it's a fight between gained horizontal resolution (3x) vs horizontal blurring to avoid colour bands.
ClearType (or that is, standard font rendering on Windows) does a lot of hinting as well, which makes the result "clearer". The 5-point fir is the general solution as far as I know, but try to play around with it! It does however have a lot to do with colour fringing, and I can definitely see it with only the 3-point solution. The 5-point should actually be more sharp than the 3-point, the key point to consider in any case is that the sum of the whole kernel must not overflow the target pixel size.
It is my intention to design the system such that filter kernels and gamma stuff and the like is modular template functors, such that it can be changed on the fly, without runtime performance hits. I'm already working on a much better design which also supports bgr/rgb/gbr on big/little endian and rgb+argb. Also I'll add support for per-monitor settings.
I found a 1-4-6-4-1 5-point kernel to be the most pleasing to my eye. I also see colour fringing with 3 point kernels, but I also see colour fringing with ClearType. the (1-4-6-4-1)/16 can be done with less multiplications than the (1-2-3-2-1)/9, because everything besides the center can be accomplished with bit-shifting and your algorithm is bit-shifting already anyway in roundedMul. I did it like this:
// 5 point lowpass filtering
const std::uint_fast32_t alpha256 = gamma(alpha)*colourSetup[3];
const std::uint8_t t04 = static_cast<std::uint8_t>((alpha256 + 0x800) >> 12); // 1/16
const std::uint8_t t13 = static_cast<std::uint8_t>((alpha256 + 0x200) >> 10); // 4/16
const std::uint8_t t2 = static_cast<std::uint8_t>((alpha256*96 + 0x8000) >> 16); // 6/16
addToAlphaMap(0, t04);
addToAlphaMap(1, t13);
addToAlphaMap(2, t2);
addToAlphaMap(3, t13);
addToAlphaMap(4, t04);
Okay, so I got some time again.
Fixexd multiple precision problems and generalized the gamma/kernel stuff, as well as the support and detection for bgr/rgb/gbr. After a couple of optimizations, it is even faster than the JUCE renderer now. Once I try out doing the actual gamma correction, I will post a newer version.
As for the 6/16 kernel, I tried out the code you posted and I must admit it's quite hard to see a difference in general. I don't claim to have 20/20 vision either, but the 6/16 definitely have more colour fringing than the 3/9, especially for squashed/overlapping glyphs, because the kernel doesn't add up evenly unless you're doing straight lines. The best example is probably here, at the "Bodoni MT Poster", just below the selection on the left :
link: http://i.imgur.com/U9TXlWF.gif
It may be a bit sharper though, I'm not sure if it's worth the price. However, one can just use whatever kernel is pleasing to them, they're both included :)
Thanks for your efforts! I'm already enjoying your renderer a great deal as you originally posted it. If it's now even faster than regular JUCE, then maybe you might want to suggest some performance patches to the regular JUCE renderer? There seems to be no reasonable explanation why subpixel rendering should be faster than normal rendering and if the regular renderer could be sped up, everyone would win.
I looked at how Cleartype renders the Bodoni MT Poster font and I see a lot of colour banding, even more than the LUT16-kernel I suggested. I do see a bit of increased sharpness with LUT16 against LUT9 in the screenshot you posted btw and yes, I do have 20/20 vision. I think Cleartype does very strong horizontal (and vertical) hinting as well as not blurring much and the result is rather sharp text with quite some colour banding.
If the goal of a subpixel renderer is for JUCE fonts to match windows fonts better and replicate the cleartype results, then we need more sharpness (and colour banding) than the LUT9-kernel. I think the LUT9-kernel from the paper and your original code looks much like what Mac OSX does (and seems to be ideal for Retina-Type display), but Cleartype is a different beast that stays reasonably sharp even if the screen is low-res.
Anyway that's all irrelevant if you make the kernel user-configurable. I just suggest you take a look at enlarged screenshots of ClearType-rendered fonts.
Important Question: Is it ok to use your renderer in a non-gpl project? I own a commercial JUCE license and would like to use it. The header comment of the code you posted says GPL. But when you initially posted the code, it felt like you could imagine the renderer to be built into JUCE and therefore have a JUCE-y license to it.
No problem, I'm really getting more and more interested in this topic - never bothered with fonts before!
Okay, so I stopped temporarily with getting the correct gamma to work and tried comparing some major textrenders to this one. I found the Castellar TrueType font to be the hardest font to get right - because of the thin lines and holes inside the glyps (so smoothed renders blurry the insides, and sharp renders look ugly hinted). Please take a look and let me know what you think:
Open it here for full resolution: http://i.imgur.com/FnFVED1.gif
My current thoughts...
- Even though the Cleartype (word) render is sharp, it is way to colourized and hinted to my taste. It also ignores a lot of the details.
- The photoshop render is not too good - looks quantized, even though it is only antialiased, it shouldn't produce 'holes' in the stems in E-glyphs, although it probably does better with the insides.
- The OSX render is overly bold as usual, however I didn't recall it being this extreme. It actually fills in nearly all insides (like, A/Z/R-glyphs), which is obviously wrong.
- The juce render - well I stated my opinions in the OP..
- The Adobe Reader is very good, nearly perfect in my opinion. It has a very good balance between hinting (although a bit too much on the vertical), resolution and still looking smooth.
- And then mine - I actually think it does a fairly good job! It is too smooth, yes, but that is the price to pay for non-hinted glyphs (use FreeType or similar together with this to get crisp fonts). However I find it to be the one to most accurately present the shape of the glyphs, if only it didn't blurry the insides..
As for performance, I don't think much can be done to the original render - much of the difference lies in it using PixelARGB::blend(), which does correct alpha/opacity blending and clamping for ARGB images. Notice that this is not possible for subpixel graphics, and the clamping is mathematically defined (not possible to overflow with alphablending), so it can skip a lot of checks. Alas, the new gamma code changes the tables again, performance lowered to again be around 20% slower.
And yes, it will get a license compatible with proprietary softwares.
I agree your renderer does the best job for this font, followed by Acrobat, so congrats! I'm surprised by the huge differences and OSX obviously does not like the font. Btw results vary between versions of OSX in my experience. Over the years, their font rendering got much quicker, but it also became a bit less accurate for extreme cases like this font.
About the performance thing.. If there are cases where the JUCE renderer could skip the correct alpha/opacity blending like your renderer, then couldn't that lead to a performance gain in those cases?
Okay, so this will be an actual working version! This is what's new:
features added (lets call this v 0.1):
weightmaps for 1/3, 5/9 and 6/16 weights.
gammascales for linear (the one juce uses), power scale (the correct) and a lut gamma scale (approximation to the former)
fixes/additional stuff:
- detection of computer endianness
- (more) correct collision
- correct offset and redrawing
- a lot of optimization
- general focus/support on making it look the same everywhere
runtime detection and support (on each seperate monitor) for:
- glyphs that span multiple monitors
- whether the system antialiases fonts
- rotated screens
- rgb/bgr/gbr/rgb displays
- scaled systems (retina, larger/lower DPIs)
- system gamma
- ARGB images (although the alpha channel is ignored)what needs to be done / happen:
- a lot of code for OSX and linux (mostly only API detections), although everything should work on windows now.
android and iOS is probably going to be ignored (but supported) since it is in their nature to rotate constantly.
- a newer version of LLVM that doesn't crash when it compiles my code.
- juce software renderers needs to know whether it is rendering onto an image, or an actual window
- juce software renderers needs to know where the origin of the window is, so it can select the correct gamma/matrix order.
- the code for rotated text isn't currently working, the positioning is correct, but the glyphs themselves aren't rotated.
Here's a link:
https://dl.dropboxusercontent.com/u/41702019/CPL.SubpixelFonts.zip
Please, if you can, test it in different environments, comment on design, bring ideas, whatever! Also, API-specific code for OSX and linux variants is much needed!
I'll probably create a repository for it soon, maybe even my whole library.. it's just very much a WIP, so, for now, it will be .zip files.
There is some parts of the code referencing non-included parts of my library, I hope they're either obvious - otherwise you should be able to find them here: http://pastebin.com/kmjgs9dM
Thanks for posting the improved version! I like all the renderer feature tweaks you did, but I don't fully agree with the logic used for display detection. As far as I oversee your code the idea is to detect display settings initially (In the Graphics constructor) and then hope the user will call the update method himself, should anything change.
However I think the renderer should react to changes to cleartype by itself without needing a call to update(). I dug though MS documentation a bit and it turns out there is a Notification that is sent to top-level windows notifying them of changes to these System Parameters. It is called WM_SETTINGCHANGE and here's a link to the document: https://msdn.microsoft.com/en-us/library/windows/desktop/ms725497(v=vs.85).aspx. This message in my opinion should automatically trigger an update of your display info data.
It turns out, JUCE already catches this message to react to display changes. It calls Desktop::Displays::refresh() and that's probably where the cleartype info should be retrieved. I guess it's not possible without changes to JUCE library code though :/.
However I don't think the subpixel renderer should have better logic than ClearType itself. ClearType currently totally ignores screen rotation (at least on Windows 8.1 where I checked) and weirdly enough I think your renderer should do the same (on windows). The reason is that users which want correct subpixel rendering, will install utilities that change cleartype settings (and send WM_SETTINGCHANGE) if the screen rotation changes. For these very subpixel aware people, your screen rotation detection will break things by double-correcting the switched order or not reacting to ClearType setting changes triggered by these helper utilities. Everyone else does not seem to care about bad subpixel order for rotated displays anyway.
Also you don't know what physical orientation the display had when ClearType was adjusted, you just have to assume the cleartype subpixel order is the way the user wants it. For example if I had my display mounted upside down for some reason, I'd set the ClearType pixel order to BGR but screen rotation woud be reported as upside-down and then the detection code would wrongly decide to use RGB rendering. Bascially the setting does not say what subpixel order the screen has, but only what subpixel order the user decided to use regardless of orientation.
No problem. The update method should be called from system events like WM_SETTINGCHANGE/WM_DISPLAYCHANGE - I just need a clever way to intercept this, haven't figured that out yet.
Also, I have not thought about the double-correcting issue.. But the better way would be to (if possible) access the device info instead, as the current code on OSX does.
But none of this will be relevant unless we get the changes posted in:
http://www.juce.com/forum/topic/suggestion-renderers-need-know-where-theyre-drawing
I'll add something about the display change notification.
Small update, the renderer now works on OSX with good results! Obviously with retina resolution, nothing looks bad, but for standard displays - here's a quick comparison I threw together:
link: http://i.imgur.com/yjcL9vu.png
As we can see, the CoreGraphics embolds the fonts quite a lot (but that should be no surprise by now), so that'll definitely be a problem if you want your GUI to look the same everywhere. As we can see on cpl renderer, nearly everything works (except rotated text ("render/utility/settings"), it is rendered by the built-in JUCE software renderer. However, more interestingly, the CoreGraphics clearly hints and snaps glyphs to integers vertically, looking at "Settings" to the left, it really distorts the spacing between letters. Probably because the text renderer has strong automatic vertical hinting, but they dont switch that around for rotated text...
New version coming up soon.. Maybe I should stop spamming this section, don't know what jules' policy is here - can someone move it to useful components/tools?
I figured Component::parentSizeChanged() always gets called for the main component if JUCE gets a WM_SETTINGCHANGE msg, so I tried updateing the renderer there. While I think this would work for a standalone application, it did not work reliably for the audio plugin I am developing. I guess that's because this message is only sent to the host window and not necessarily to the subwindow my gui lives in. So now I'll just poll the settings once or twice per second. I believe the performance cost is negligible.
Depends.. On Windows, just getting the screen rotation took around ~20 ms iirc.
Oh ok.. that's annoying and surprisingly slow. What version of windows did you measure? I'm still using your old version with my own detection code that only detects subpixel order, not screen rotation and I found polling the ClearType settings with SystemParametersInfo is rather fast both on Windows 7 and Windows 8. However of course there's no guarantee about the time it will take :(.
It's Windows 7.
As we speak I'm finishing up a much better system using events. On OSX, one can use CGDisplayRegisterReconfigurationCallback() to register a callback on any display change, while on Windows I'm using SetWindowsHookEx() with WH_CALLWNDPROCRET, GetModuleHandle(0) and GetCurrentThreadId() (ensure you're on the main thread when attaching the hook). It may seem kinda grim to use hooks, but it is actually a pretty good way to solve the problem: it's safe, lightweight and customizable. And of course, it's 100% reliable.
Hi Mayae,
Trying to finally finish the plugin I'm working on, I switched to full optimization in MSVC Express 2013 and that for me unfortunately breaks the latest version of the renderer you posted here. I'm not sure what setting causes the problem (yet), but I wonder what version of MSVC you use. Did you try full optimization? I believe I enabled everything for best speed and the result I get is that many strings do not render at all and some render without any subpixel colors and with the colour wrong.
By compiling many times I figured the setting causing the issue is \Oi (Generate intrinsic functions). It is part of \O2 and \Ox, but not \O1 and it does use special x86 opcodes where possible to speed up execution. I have no idea how it can affect your code.
Hello guys,
I have been following this post and it looks very promising.
I am no expert on font rendering so I wanted to ask if the code can be used to render text to an image or if it depends on being rendered to a display.
Thanks
Dan