Thai Input problem with TextEditor, I made it works

Linux
1

Dear all.

I have just discovered JUCE two days ago, and I’m quiet amazed.
Following the tutorials and looking at the demo and project files I manage to write a small demo.

I read that JUCE was unicode and in facts, I can display my thai language as wanted. Either in label, drawFittedText or TextEditor. It is well displayed. To display the text, i switch my keyboard to thai, and I write the content directly in my .cpp file with emacs which save the file in unicode, then I compile, then run my application. The thai is displayed as wanted.

The problem is when I try to edit thai language in my TextEditor which is able to dispaly thai as initial text (by setting the right font), I jus’t can’t. At best he display the ASCII character which is on the same keyboard key as the thai character although my keyboard is switched to thai.

Even if i try to copy from my initial TextEditor display and paste to the TextEditor, it doesn’t work.

Does anybody has idea about that ?

Thanks.
Christian.

2

Thanks - glad you’re enjoying it!

Must be the linux key input code that’s not handling unicode properly… I’ll make a note to look at this when I get chance, but if you want to check it out yourself, the place to look will be in juce_linux_Windowing.cpp

3

Thanks Jules, but I think it will be difficult for me(first time i did GUI) to solve the problem, but i won’t give up (open source power;-)).

Here some interesting things :

In my application I read a file, and i display the content of the file with the function loadFileAsString in the paint function provided with the demo. Everything display well.

I run my app, and I also run Yudit, in which i edit this file, i write some thai character, then I save the file as UTF-16 and magic the new content is displayed in the juce app.

The interesting thing is how I can use Yudit to input my thai, here only Yudit is involved :

I have 4 ways :

  • Straight : I think in this mode, Yudit just get whatever is pressed from the keyboard, so if i set my xfce4-xkb-plugin, (which is initialised with setxkbmap fr,th), to french, a key press display the correct french char, but if it is set to thai, he displays the thai character as wanted.
    (It is the way I try to edit text in TextEditor)

  • Thai input : if I use that method i don’t need to use my xfce4-xkb-plugin, switching it to thai to enter thai, (Yudit does the thing itself) the only problem is that all the character doesn’t match what I expected, it looks similar similar like using qwerty instead of azerty. (diiferent map)

  • Unicode : if I use that method, if my xkb-plugin keyboard is set to french, i get what i want at least for ASCII char, get problem with accent, double char are displayed. if I try to enter thai, the same result. Each keystroke make appears two characteres, which doesn’t have glyph.

But if I hold on the key ALT while pressing my thai key, I got it right. if I hold on the ALT key while pressing a standard ASCII key is pressed, nothing is displayed, the cursor acknowledge the key stroke but it just doesn’t move.

  • TIS-620 : Nearly the same as unicode.

CONCLUSION :
I want to know if the following things that I’m thinking are correct :

  • Is JUCE TextEditor is expecting UNICODE as input method ?

  • The X keyboard doesn’t seem to use UNICODE otherwise Yudit should get it right while the UNICODE input method is used. shouldn’t need the ALT key.

  • Is the ALT button has something to do with the fact that i can’t input my thai text in TextEditor (The french accent which are one key pressed can be input, but not character that need the ALT GR to be hold on, nor any other combining accent like ï)

  • Maybe the ability to press ALT and my Thai char should work for TextEdit, but it doesn’t, nothing happens, it just ackowledge the keystroke and the the cursor doesn’t move.

I am a bit confused and I hope this remark can help.

OK I have Yudit source code, JUCE source Code, Xlib Source Code so i feel abit ashamed to ask for help :wink:

Christian.

PS : Some of my LOCAL are set to fr_FR.UTF-8 and some to en_EN.UTF-8, I just told that if he can help.

4

The previous post is nearly deprecated.

I downloaded latest version of Yudit and now the only way I can input thai is :

Yudit Input method x-utf-8:en_GB : if my xfce-xkb plugin is in french, i write french letter, if it is in thai, i write my thai letter.

The Thai input method : It doesn’t match my keyboard but the character are well displayed (still the same as before).

All the previous input : unicode, TIS-620, Straight doesn’t work anymore.

I’m bit lost, since I don’t have a clue about X key things, but I’m learning step by step.

Christian.

5

Hi,

after reading some stuff about xlib and XKBLib, I finally find a way to input my thai.

Here is what I have done but I think it’s ugly and he needs to be rewritten carefully, it may be a help to input other languages similar to thai.

The first thing I have done is in juce_linux_Windowing.cpp :

  • in handleWindowMessage, for the case KeyPress,
    I replace
    KeySym sym = XKeycodeToKeysym (display, keyEvent->keycode, index);
    by
    char asci[64]; // Must not be above 64.
    KeySym sym;
    XLookupString(keyEvent, asci, sizeof(asci), &sym, 0);
    (Found this in yudit source code)

(XLookupString seems to be powerful and take care of group(language map: french and thai in my case) and level (shift, ctl, …))

if sym represent a thai caractere his value range vary from
XK_Thai_kokai 0x0da1 to XK_Thai_lekkao 0x0df9

asci is a 3 bytes array if printed with printf from a utf-8 console it will display my thai caracter.

The problem I had is that i couldn’t give sym to handlePressKey because of the function
insertTextAtCursor (String::keyCodeToString (key.getKeyCode()));

here keyCodeToString expect a keyCode that is coded only in one byte.
The problem is that I didn’t know how to convert the sym into the asci array. If I knew I had rewritten this function.

So I realised that if I pass asci to the function
insertTextAtCursor (); in TextEditor, it will display my charactere well.

Now the ugly thing happens :

  • I modified the Class KeyPress to have a string member.
    (First I wanted to create a class KeyPressExt with inheritance from KeyPress it didn’t work)
  • So I had function like getStr, setStr and a couple of constructor that take a String as a parameter.

So in the file juce_linux_Windowing following the above addition I added
String str = String(asci);
KeyPress kpe = KeyPress((int) sym, 0, str);

Then I added a test to call handleKeyPressExt only of my charactere typed is thai

if( (sym >= XK_Thai_kokai) && (sym <= XK_Thai_lekkao) ){
keyPressed = false; //make sure the other things don’t happen
printf(“sym is a thai char!\n”); // This for debug.
// Use our own KeyPress handling
handleKeyPressExt(kpe);
}

Then I also added the function handleKeyPressExt which is as following :

void handleKeyPressExt (const KeyPress keyPress){
updateCurrentModifiers();

if (Component::currentlyFocusedComponent->isValidComponent())
  Component::currentlyFocusedComponent->keyPressed(keyPress);
else
  component->keyPressed(keyPress);

}

The aim of that function was to skip the call to internalPressKey since I had already created my KeyPress object.

Then in TextEditor I added to the function keyPressed :

if (! isReadOnly())
if ( (key.getKeyCode() >= XK_Thai_kokai)
&& (key.getKeyCode() <= XK_Thai_lekkao))
insertTextAtCursor (key.getStr());
else{
insertTextAtCursor (String::keyCodeToString (key.getKeyCode()));
}
lastTransactionTime = Time::getApproximateMillisecondCounter();

I did that to be careful, but i think I just could call getStr whatever is the keyCode.

To conclude what should be done :

  • use XLookupString instead of XKeyCodeToKeysym
  • Modify the keyCodeToString Function to handle value of keycode above 255.
  • I think this would make it works for many languages similar to thai (thai, lao, indic, khmer, …)

To go further maybe we should use XIM method to really get a multilanguage capable library.

6

Great stuff. Thanks for finding out all that info.

I think the solution might actually be a lot simpler than what you’ve done there - it’s already possible to pass a unicode character to the handleKeyPress method. Try this:

[code] void handleWindowMessage (XEvent* event)
{
switch (event->xany.type)
{
case 2: // ‘KeyPress’
{
XKeyEvent* keyEvent = (XKeyEvent*) &event->xkey;
updateKeyStates (keyEvent->keycode, true);

            char utf8 [64];
            zeromem (utf8, sizeof (utf8));
            KeySym sym;
            XLookupString (keyEvent, utf8, sizeof (utf8), &sym, 0);
            int keyCode = (int) String::fromUTF8 ((const uint8*) utf8, sizeof (utf8) - 1) [0];

            if (keyCode < 0x20)
                keyCode = XKeycodeToKeysym (display, keyEvent->keycode, 
                                            (currentModifiers & ModifierKeys::shiftModifier) != 0 ? 1 : 0);

            const int oldMods = currentModifiers;
            bool keyPressed = false;

            const bool keyDownChange = (sym != NoSymbol) && ! updateKeyModifiersFromSym (sym, true);

            if ((sym & 0xff00) == 0xff00)
            {
                // Translate keypad
                if (sym == XK_KP_Divide)
                    keyCode = XK_slash;
                else if (sym == XK_KP_Multiply)
                    keyCode = XK_asterisk;
                else if (sym == XK_KP_Subtract)
                    keyCode = XK_hyphen;
                else if (sym == XK_KP_Add)
                    keyCode = XK_plus;
                else if (sym == XK_KP_Enter)
                    keyCode = XK_Return;
                else if (sym == XK_KP_Decimal)
                    keyCode = numLock ? XK_period : XK_Delete;
                else if (sym == XK_KP_0)
                    keyCode = numLock ? XK_0 : XK_Insert;
                else if (sym == XK_KP_1)
                    keyCode = numLock ? XK_1 : XK_End;
                else if (sym == XK_KP_2)
                    keyCode = numLock ? XK_2 : XK_Down;
                else if (sym == XK_KP_3)
                    keyCode = numLock ? XK_3 : XK_Page_Down;
                else if (sym == XK_KP_4)
                    keyCode = numLock ? XK_4 : XK_Left;
                else if (sym == XK_KP_5)
                    keyCode = XK_5;
                else if (sym == XK_KP_6)
                    keyCode = numLock ? XK_6 : XK_Right;
                else if (sym == XK_KP_7)
                    keyCode = numLock ? XK_7 : XK_Home;
                else if (sym == XK_KP_8)
                    keyCode = numLock ? XK_8 : XK_Up;
                else if (sym == XK_KP_9)
                    keyCode = numLock ? XK_9 : XK_Page_Up;

                switch (sym)
                {
                    case XK_Left:
                    case XK_Right:
                    case XK_Up:
                    case XK_Down:
                    case XK_Page_Up:
                    case XK_Page_Down:
                    case XK_End:
                    case XK_Home:
                    case XK_Delete:
                    case XK_Insert:
                        keyPressed = true;
                        keyCode = (sym & 0xff) | extendedKeyModifier;
                        break;
                    case XK_Tab:
                    case XK_Return:
                    case XK_Escape:
                    case XK_BackSpace:
                        keyPressed = true;
                        keyCode &= 0xff;
                        break;
                    default:
                    {
                        if (sym >= XK_F1 && sym <= XK_F12)
                        {
                            keyPressed = true;
                            keyCode = (sym & 0xff) | extendedKeyModifier;
                        }
                        break;
                    }
                }
            }

            if ((sym & 0xff00) == 0 && sym >= 8)
                keyPressed = true;

            if (oldMods != currentModifiers)
                handleModifierKeysChange();

            if (keyDownChange)
                handleKeyUpOrDown();

            if (keyPressed)
                handleKeyPress (keyCode);

            break;
        }

[/code]

and I also changed line 177, just in case it interferes with extended unicode chars:

static const int extendedKeyModifier = 0x10000000;

7

Hi, Jules.

I downloaded the library, inserted your code, compile, …, didn’t work, i
just needed to had this line :

if( (sym >= XK_Thai_kokai) && (sym <= XK_Thai_lekkao) )
		  keyPressed = true;

before (or after) that

if ((sym & 0xff00) == 0 && sym >= 8)
		  keyPressed = true;

Now it works just perfectly so that I will be able to insert thai language in application. I still have a small problem but it is not urgent and can wait is that in French I can’t input compound caracters such as ï (" i)or Ô(^O).

Anyway I’m already happy with what JUCE can do now, and it has been with great pleasure that I helped as I could.

Thanks.
Christian.

8

That’s good, but I’m not happy about hard-coding any thai characters in there - how about this tweak instead:

                if (utf8[0] != 0 || ((sym & 0xff00) == 0 && sym >= 8))
                    keyPressed = true;
9

Hi,

I just made that last change, it works perfectly. It’s even better, I didn’t like the idea to put thai caractere there too, but to be honest i don’t really understand what is the meaning of that line

((sym & 0xff00) == 0 && sym >= 8)).

Many Thanks, now i think it will work with many other languages as well.

Christian.

10

Great, glad it’s working now!

11

summoning again this post, as i’ve encounter some problems on system with kernels compiled with default charset as UTF-8, i’m unable to type accented characters.

the fix is easy, we should trick X11 to use the default locale when processing a keypress. i don’t know if this could be done system wise one and for all in doPlatformSpecificInitialisation without broke anything else. so i came up with a RAII class to swap the locale when processing key events in windows.


//==============================================================================

class ScopedLocaleChange
{
public:
    ScopedLocaleChange()
        : oldLocale (::setlocale (LC_ALL, 0))
    {
        ::setlocale (LC_ALL, "");
    }
    
    ~ScopedLocaleChange()
    {
        ::setlocale (LC_ALL, oldLocale);
    }
    
private:
    const char* oldLocale;
};

// ...

    void handleWindowMessage (XEvent* event)
    {
        switch (event->xany.type)
        {
            case 2: // 'KeyPress'
            {
                ScopedXLock xlock;
                ScopedLocaleChange localeChange;

in this way we are sure Xlib keysyms are translated correctly with all locales.

12

ok… presumably this would have the same effect:

[code] char utf8 [64];
zeromem (utf8, sizeof (utf8));
KeySym sym;

            {
                const char* oldLocale = ::setlocale (LC_ALL, 0);
                ::setlocale (LC_ALL, "");
                XLookupString (keyEvent, utf8, sizeof (utf8), &sym, 0);
                ::setlocale (LC_ALL, oldLocale);
            }[/code]

Normally I love RAII, but I don’t like the idea of that char* being kept and re-used silently at some random point later on… And since this is basically an dodgy old-style C hack, I think it’s better to keep the machinery of the hack obvious - if you hide it all in an elegant c++ structure, it’d be easy to look at this function in the future and not notice that the locale is being messed about with.

13

yes right !