Bug when setting Japanese text to TextEditor

General JUCE discussion
1

Hi, JUCE developers.
I found a bug in TextEditor that a part of the string is not drawn when a string containing double-byte spaces is set and highlighted.
This bug was confirmed in JUCE 6.1.2.

Here are sample codes.

<MainComponent.h>

#pragma once

#include <JuceHeader.h>

class MainComponent  : public juce::Component
{
public:
    MainComponent();
    ~MainComponent() override;

    void paint (juce::Graphics&) override;
    void resized() override;

private:
    juce::TextEditor editor;

    JUCE_DECLARE_NON_COPYABLE_WITH_LEAK_DETECTOR (MainComponent)
};

<MainComponent.cpp>

#include "MainComponent.h"

using namespace juce;

// Note: japanese charactor and multi-byte space are repeated
// あ い う え お あ い う え お あ......
const auto japanese_text(CharPointer_UTF8("\xe3\x81\x82\xe3\x80\x80\xe3\x81\x84\xe3\x80\x80\xe3\x81\x86\xe3\x80\x80\xe3\x81\x88\xe3\x80\x80\xe3\x81\x8a\xe3\x80\x80\xe3\x81\x82\xe3\x80\x80\xe3\x81\x84\xe3\x80\x80\xe3\x81\x86\xe3\x80\x80\xe3\x81\x88\xe3\x80\x80\xe3\x81\x8a\xe3\x80\x80\xe3\x81\x82\xe3\x80\x80\xe3\x81\x84\xe3\x80\x80\xe3\x81\x86\xe3\x80\x80\xe3\x81\x88\xe3\x80\x80\xe3\x81\x8a\xe3\x80\x80\xe3\x81\x82\xe3\x80\x80\xe3\x81\x84\xe3\x80\x80\xe3\x81\x86\xe3\x80\x80\xe3\x81\x88\xe3\x80\x80\xe3\x81\x8a\xe3\x80\x80\xe3\x81\x82\xe3\x80\x80\xe3\x81\x84\xe3\x80\x80\xe3\x81\x86\xe3\x80\x80\xe3\x81\x88\xe3\x80\x80\xe3\x81\x8a\xe3\x80\x80\xe3\x81\x82\xe3\x80\x80\xe3\x81\x84\xe3\x80\x80\xe3\x81\x86\xe3\x80\x80\xe3\x81\x88\xe3\x80\x80\xe3\x81\x8a\xe3\x80\x80\xe3\x81\x82\xe3\x80\x80\xe3\x81\x84\xe3\x80\x80\xe3\x81\x86\xe3\x80\x80\xe3\x81\x88\xe3\x80\x80\xe3\x81\x8a\xe3\x80\x80\xe3\x81\x82\xe3\x80\x80\xe3\x81\x84\xe3\x80\x80\xe3\x81\x86\xe3\x80\x80\xe3\x81\x88\xe3\x80\x80\xe3\x81\x8a\xe3\x80\x80"));

MainComponent::MainComponent()
{
    editor.setMultiLine(true);
#if JUCE_WINDOWS
    const Font font("MS UI Gothic", 15.f, Font::plain);
#elif JUCE_MAC
    const Font font("Arial Unicode MS", 15.f, Font::plain);
#else
    const Font font("IPAGothic", 15.f, Font::plain);
#endif
    editor.setFont(font);
    editor.setText(String(japanese_text));
    editor.selectAll();
    addAndMakeVisible(&editor);

    setSize(300, 400);
}

MainComponent::~MainComponent()
{
}

void MainComponent::paint (juce::Graphics& g)
{
}

void MainComponent::resized()
{
    editor.setBounds(getLocalBounds());
}

When you run this code, you will see the following:
image

then unhighlighted,
image

Here is a sample project file.
TextEditorBugSample.zip (9.1 KB)

I investigated the cause of the problem and found the following things wrong.

In TextEditor::UniformTextSection::initialiseAtoms(),

...
            // create a whitespace atom unless it starts with non-ws
            if (text.isWhitespace() && *text != '\r' && *text != '\n')
            {
                do
                {
                    ++text;
                    ++numChars;
                }
                while (text.isWhitespace() && *text != '\r' && *text != '\n');
            }
...

This implemention is that only one-byte spaces are considered as spaces, and multi-byte spaces are not judged as spaces.
More detail, CharPointer_UTF8::isWhitespace() is false when first charactor is multi-byte space.

And, other implemention in TextEditor is called CharacterFunctions::isWhitespace().
That function is true when first charactor is multi-byte space.
So, this strange behavior has occured.

I hope that they will handle this bug.

Thanks.

2 Likes
2

Thank you for reporting this and also for finding the root cause of the problem.

I made a patch that fixes the problem you mentioned along with some other, that also affects typing in multi-keystroke layouts. I don’t use such layouts myself, so while it looks okay to me, I would appreciate if you could take a look and see if it behaves as you would expect.

texteditor_fix.patch (2.5 KB)

1 Like
3

Thank you for investigating the problem and creating the patch.
I checked it in my Japanese Windows environment, and the problem has been resolved.

Thanks.

4

@attila
Will this fix be in the next JUCE version?

5

The change is now out on develop, and will be part of the next point release.

6

Thanks for your commit!