Can't input non-English characters in TextEditor


#1

Hello.

I'm new to JUCE and want to know:

1. How to input and display non-English characters in a textEdit? Or How to set and use the OS' default font with JUCE components?

2.  On MAC OSX, the characters/text within the UI of all JUCE application looks fine, but on Windows and Ubuntu-Linux, they look fuzzy, seem a little bit ugly and strange. How to make them clear, just like the OS' default UI characters?

Thanks in advance.


#2

add:

On Ubuntu-Linux, can't input any non-English in a textEdit, it just display English characters however I switch IMEs.


#3

Anybody can help?


#4

On windows the fonts in JUCE look a bit more fuzzy because Windows usually uses more agressive hinting for fonts.

And inputting 你好 with the IME in a text field will work, but the font we're using doesn't contain those characters so they come up as boxes, or they are not visible at all, depending on if you use the DirectWrite text renderer or not (tested on Windows 7).

Finding out the default UI font on Windows is a bit complicated:

1) you have to use the right API function. Windows has a few legacy API functions, which return some default font used in various obsolete versions of Windows. The call you need is

// Please note I left out any error checking!
NONCLIENTMETRICS ourMetrics = {sizeof(ourMetrics)};
SystemParametersInfo(SPI_GETNONCLIENTMETRICS, sizeof(ourMetrics), &ourMetrics, 0);

and then you can get the font name from the lfMessageFont member and use it as the default font name for a juce::Font. The height value is set to a negative number, indicating it is the character height of the font—see point (2) for that. The height is returned in pixels.

A common default for western languages is Segoe UI, 12px, at 96DPI, corresponding to the 9pt you see in the Window Color and Appearance control panel. The x-height of this font for this size is 6px.

2) The font height in JUCE and the font size in Windows have a different meaning. In Windows the font height often refers to the character height, which excludes any extra space for accents etc. This extra space is known as the internal leading. Take a look at the scheme here: https://support.microsoft.com/en-us/kb/32667.

In JUCE the font height always includes the internal leading. Some more calls to the API reveal this extra spaces amounts to 3px:

HDC dc = GetDC(0);
SelectObject(dc, (HGDIOBJ)CreateFontIndirect(&ourMetrics.lfMessageFont));
TEXTMETRIC ourFontMetrics;
GetTextMetrics(dc, &ourFontMetrics);

So the font height to use in JUCE is 15px.

As a consequence, if you want to use the default font size, you can't use hard-coded font sizes in your application.

3) If you're using a higher DPI setting, the API will return larger pixel sizes, which you have to scale down to what it would be at 96dpi.

4) So how do you actually use that font in your JUCE program? In my experience JUCE will make your life fairly miserable if you want to stray from the default font. Some of the default fonts for components can be overridden in your LookAndFeel, for the others you can set the font explicitely via the component (for instance TextEditors). For still others you'll have to override the paint() method. I haven't found an elegant way to override the font for all the components in your application.

Final note: What is the default font in JUCE on Windows?

JUCE uses Verdana and Tahoma. Those fonts were designed to be legible at a small pixel size, owing to the low resolution of monitors at the time (around 1995). They have short ascenders and descenders compared to the x-height. Back then the default font size would be 8pt, which translates to 11px, and an internal leading of 2px. And thus a font height in JUCE of 13px. Once again we have an x-height of 6px.

The default height is 14px, and you may have noticed it indeed appears a bit larger than the default font in Windows (it is visible in the menu of the Introjucer).


#5

Hi, roeland, thanks for your reply.

It's a great helpful for me, although I can't understand these completely. These things seem more complicated than I thought. I've tried my best doing some research and experiments, but it's too difficult for me to have any luck. BTW, I hate the capital words of the Microsoft's. :) and I confirm that JUCE doesn't support IMEs on Linux, and Jules has no interesting to work it out include the Font and clearType things, which makes me sad...

However, I'll do more learning and researching, hope one day I can solve these problems, but I'm not sure I'll have the abilities and enough time...

Thanks again.


#6

The easy version is just getting the name of the default font, and use that.

NONCLIENTMETRICS ourMetrics = {sizeof(ourMetrics)};
bool success = (bool) SystemParametersInfo(SPI_GETNONCLIENTMETRICS, sizeof(ourMetrics), &ourMetrics, 0);
if (success) {
    String fontName =  CharPointer_UTF8(ourMetrics.lfMessageFont.lfFaceName);
    // and use the name...
}

Edit: using CharPointer_UTF8 is incorrect, see reply below.

The default font size is 14px, which for "Segoe UI" will result in slightly smaller text than the default. The Chinese characters still don't display properly though, but maybe on Chinese languate systems the default font is different. If on Chinese versions the default font is the same you will not solve the display problem by using a different font.

How to support using an IME on Linux—I have no idea. There's a good chance that all those desktop environments (GNOME, LXDE, KDE, Xfce) all have their own unique API, in which case the chances of having it supported in JUCE are 0.

 → side note: a lot of stuff seems obvious on Windows or Mac, but is not fully support on Linux systems. Examples include copy/paste of anything else than text (breaks when you copy and then close the app) and transparent windows (on a lot of systems only 1-bit matte is supported via some hackery). Until recently using 2 monitors with different resolutions was also broken (resulted in a corner above or below the smaller screen where the mouse cursor and windows could "disappear").

--
Roeland


#7

Sorry, but IME on Linux is currently not yet supported. See this thread: http://www.juce.com/forum/topic/ibus-ime-support


#8

Thanks roeland!

I've tried the code on Windows 7 (Chinese version), it didn't work. I don't know how to describe it exactly in English... The windows' 'lfFaceName' got the right font name, but the JUCE String object 'fontName' couldn't identify it correctly, it was a '?', so all the application's UI characters of Non-English were just showed '□□□'...

Please see the screenshort below:


#9

Thanks for your reply, timur.

Does this mean  that we have to give up using JUCE for all app which involve text input on Linux? If so, It is to be regretted.


#10

You may need to use CharPointer_UTF16 instead of CharPointer_UTF8 if you build your application with unicode enabled. Now that I think of it, not sure if it's possible to get UTF-8 encoded text via the Windows API.

Anyway your screenshot shows fontName definitely doesn't contain "?" (you pointed this out yourself on the screenshot). Not sure why the output doesn't show the name.

 

Edit: using CharPointer_UTF8 is incorrect, see my comment below (or on next page). For non-Unicode builds, use the MultiByteToWideChar function to convert to UTF-16


#11

Just for anyone reading this, the comment above it not correct! 

JUCE Strings can use UTF8, 16 or 32 and it'll all work exactly the same way, being converted internally for things like win32 calls.


#12

Yes, roeland. I've tried all the way I could think of to obtain the correct result before I took the screenshort, such as UTF8, 16, 32, and std::string, windows CHAR[32] covert, etc., it all didn't work (the JUCE String object fontName just showed '?'), or it even couldn't compile.


#13

Not really, Jules. You may test it on Non-English OS to observe the result if you have time and interesting. For example, the codes which roeland posted. And it's sure that JUCE and Introjucer handling Non-English things isn't good enough. For examaple, if source files contain Non-English charaters (such as comments), when save them through Introjucer and then compile on Windows VS, it cannot compile, you must covert the files to UTF-8 format (with sign) one by one... 

Anyway, JUCE still is a great Library. Thanks for your grand work.


#14

Windows has a 'character set' option, which can be set to Unicode and a Multibyte.

If you look at the LOGFONT definition it contains:

TCHAR lfFaceName[LF_FACESIZE];

TCHAR is defined to be a char for a build using Multibyte, and to a wchar_t when using Unicode. So you need to use the appropiate function in JUCE to convert that to a JUCE string using the appropiate function. Now that I think of it, using CharPointer_UTF8 is wrong.

For an Unicode build (which is the more sane option, IMO) it's easy—use CharPointer_UTF16.

For a Multibyte build, the string is encoded in whatever code page your application happens to be using, and you need to convert the string to UTF-16 with MultiByteToWideChar(CP_ACP, 0, …), and then again use CharPointer_UTF16.

Introjucer leaves this setting empty, so I don't know what you actually will end up using.

--
Roeland


#15

It works on Windows!!! (using the default font of Windows)!!!laughyesyes

Here are my codes:

char* fontName_char = ourMetrics.lfMessageFont.lfFaceName;
int charSize = MultiByteToWideChar (CP_ACP, 0, fontName_char, -1, NULL, 0);

wchar_t* fonatName_wchar_t = new wchar_t[charSize];
MultiByteToWideChar (CP_ACP, 0, fontName_char, -1, fonatName_wchar_t, charSize);

String fontName = CharPointer_UTF16 (fonatName_wchar_t);
// DBG (fontName);
laf->setDefaultSansSerifTypefaceName (fontName);

delete[] fonatName_wchar_t;

Thanks very much, roeland!

And, it's so difficult to understand the things of Microsoft's, I hate them...

Now, other OSes need to be researched. May I ask, do you have any guidance or clue about these things?


#16

Well, to be fair, the introjucer's code editor was only designed to handle utf-8 or utf-16 source files.

If you're writing non-ascii comments in your code then utf-8 is probably the most sensible format for saving your files, as it's guaranteed to work interchangably in any editor and any compiler, on anyone's machine. If you save using a local codepage encoding then many other editors will struggle with it too.


#17

Yes, you're right, Jules. I know this's nothing about JUCE.

The problem is, I have never saved any source file manually except work in Introjucer. Whenerve I created and saved source files through Introjucer,  then gave some non-English comments  to them within IDEs and compile them, it's fine with Xcode, but the stupid Winodws VS coverted them surreptitiously into non-UTF-8. Then, I reopen them through Introjucer, save all, Introjucer convert them agian into UTF-8 (without sign, BOM? I don't know exactly what it's called in English), then compile within VS, it'd give wrong informations, I'll have to save them one by one as UTF-8 with sign (BOM?).  If I reopen them once more in Introjucer, changed something (unfortunately) and save, all the troubles above will come again...

There is no such troubles within Xcode.

What I mean is, wouldn't it be more convient that when Introjucer save these source files which contain non-English characters, it could save them as UTF-8 with sign? (BOM?)