JUCE Accessibility on `develop`

For people new to supporting accessibility like myself, it would be appreciated if you could elaborate on the ‘why’ you would provide AccessibilityRole (or really, how it’s useful), and in which circumstances it’s applicable (only when providing new Component subclasses?). Basically I don’t understand its purpose in code nor in an accessibility context, and can’t infer its implications from a UI point of view.

Maybe just some documentation on the class would point me in the right direction? So far it only describes the ‘what’ which isn’t really all that helpful.

Unless I’m mistaken or have missed it, is there not support for accessibility when dragging and dropping? This is a fairly common UI interaction, particularly outside of audio apps - so I’m sure having some kind of state change handling for drag sources and drop targets would be useful.

The role will be used by the screen reader client to determine some basic things like what gets read out when the element is focused and we use the roles internally to enable/disable some behaviours such as editable text, selection etc.

You should only need to be concerned with custom roles, behaviours and AccessibilityHandler subclassing if you are writing new UI elements which aren’t derived from the standard JUCE widgets as they will handle this. We’ll be pushing out an accessibility demo project soon which will have some examples of custom widgets and should make this a bit clearer.

This isn’t implemented currently, but it’s something we’ll be looking at alongside mobile support.

1 Like

Regarding PopupMenu (for example in the juce WidgetsDemo , Menus tab) with VoiceOver activated:

When a menu is freshly opened, if the mouse cursor does not happen to be over the menu window, ctrl-option-left / right does nothing, is that expected ?

If I add a submenu to the PopupMenu, ctrl-option-spacebar does not open the submenu, while VoiceOver says we can press that to select the item.

(note: I think submenus should be added in the DemoRunner demos, as they have a complex behaviour and have sometimes some subtle issues)

Also: when a menu is freshly opened, the initially selected item is the one under the mouse cursor. I wonder if that behaviour should not be changed in order to make the menus more keyboard & accessibility friendly.

1 Like

I have an about / help window that only shows static text, so I passed it through setTitle() and set a staticText role, but Narrator stops reading after the 1000th character. The same happens if I use a read-only TextEditor.

1 Like

When the next version of JUCE is released, we’re planning to update it in a series of fairly big projects, because we need some of the fixes that have been added in the mean time on develop.
Unfortunately, at this moment we don’t have the time to test whether accessibility has issues with those projects. So, instead of giving the impression to users that we half-baked it in, we would prefer to opt-out of it until a later time.
What’s the easiest way to do so?

Calling Component::setAccessible (false) on a top-level component will disable accessibility for it and any sub-components.

2 Likes

So, reframing: there seems to be no straightforward way to give Narrator a read-only text of more than 1000 characters, neither as title, description or help text. This can be verified by setting a read-only TextEditor with a text longer than that. The alternative is to implement AccessibilityTextInterface. Two issues remain:

  • Because AccessibilityTextInterface doesn’t indicate if the text (range) is read-only, Narrator says “editing”, which is misleading. Currently this can only be fixed by implementing a dummy AccessibilityValueInterface for isReadOnly().

  • If the text doesn’t end in whitespace or newline, Narrator doesn’t read the last word. This can be verified on a normal, editable TextEditor.

There are also some issues with TextEditorAccessibilityHandler, like indicating “end of line” for every character in the first paragraph, or this happening to the cursor

when a non-wrapping editor is horizontally scrolled.

    AccessibleState getCurrentState() const override
    {
        auto state = AccessibilityHandler::getCurrentState();

        if (button.getClickingTogglesState())
        {
            state = state.withCheckable();

            if (button.getToggleState())
                state = state.withChecked();
        }

        return state;
    }

It’s kinda annoying that buttons only tell you their state if getClickingTogglesState(). I have lots of buttons I manually change the state for.

I don’t understand the new focus container mechanism. Previously, if a Component was set as a focus container, the tab order would cycle within its children and never fall out of the container. This doesn’t work with UI automation -tab is ok, but if I navigate with arrow up/down, I fall out of the container. I’ve tried a lot of things and some of them seem to fix it but they all seem rather arbitrary -is there anything else, something additional we have to do now?

Thanks, the fix for this requires some fairly heavy refactoring of how macOS handles offscreen rows for tree and lists so it’ll be out sometime next week.

VO+shift+down will "enter’ the menu window and you can then navigate with VO+arrow keys.

Invoking the “show menu” command with VO+shift+m will show the submenu, or navigating with the arrow keys without VO modifiers when the menu has keyboard focus.

These are both good suggestions, I’ll look at adding them.

Thanks, I’ll look into the Narrator text interface issues next week.

The container hierarchy works slightly differently on Windows to macOS and will present UI elements as a flat tree, you grab the tab focus in a component by setting its focus container type to FocusContainerType::keyboardFocusContainer and it will behave as before.

I think that’s what I’m doing. Here is a minimal example.

struct Overlay : juce::Component
{
    Overlay()
    {
        setFocusContainerType (FocusContainerType::keyboardFocusContainer);
        exit.onClick = [this] { setVisible (false); };
        addAndMakeVisible (exit);
    }

    void paint (juce::Graphics& g) override
    {
        g.fillAll (juce::Colours::lightgrey);
    }

    void resized() override
    {
        exit.setBounds (getWidth() / 2 + 20, 20, getWidth() / 2 - 40, getHeight() - 40);
    }

private:
    juce::TextButton exit{ "exit" };
};

struct MyAudioProcessorEditor : juce::AudioProcessorEditor
{
    MyAudioProcessorEditor (MyAudioProcessor& p) : AudioProcessorEditor{ p }
    {
        enter.onClick = [&]
        {
            overlay.setVisible (true);
            overlay.grabKeyboardFocus();
        };
        addAndMakeVisible (enter);
        addChildComponent (overlay);
        setSize (600, 400);
    }

    void paint (juce::Graphics& g) override
    {
        g.fillAll (juce::Colours::darkgrey);
    }

    void resized() override
    {
        enter.setBounds (20, 20, getWidth() / 2 - 40, getHeight() - 40);
        overlay.setBounds (getLocalBounds());
    }

private:
    Overlay overlay;
    juce::TextButton enter{ "enter" };
    JUCE_DECLARE_NON_COPYABLE_WITH_LEAK_DETECTOR (MyAudioProcessorEditor)
};

tab navigation is ok, but when Narrator is in scan mode (Caps+space), you can navigate with arrow up/down and reach controls outside the focus container -in this case, the “enter” button when it’s not visible, behind the overlay.

(edit) I tried making all the overlays modal (which wasn’t necessary before) -it’s the same. UI automation can interact with components behind modal components.

A tester has reported to me a crash in juce_mac_Accessibility.mm line 67

Holder element ([cls.createInstance() init]);

It happens during application startup. He is on macOS 10.9 so that’s probably the cause of the crash.

I was trying to approach the navigation issue with a variable (ignored/group) role and handler invalidation, and I ran into this glitch: when Narrator is active, and the top layer is both shown and hidden using the mouse, automation focus gets stuck on a hidden component. This would be a simple case:

struct Container : juce::Component
{
    Container()
    {
        setFocusContainerType (FocusContainerType::keyboardFocusContainer);
    }

    std::function<bool()> ignore;

    std::unique_ptr<juce::AccessibilityHandler> createAccessibilityHandler() override
    {
        auto role{ ignore() ? juce::AccessibilityRole::ignored : juce::AccessibilityRole::group };
        return std::make_unique<juce::AccessibilityHandler> (*this, role);
    }
};

struct MyAudioProcessorEditor : juce::AudioProcessorEditor
{
    Container container1, container2;
    juce::TextButton btn1{ "open" }, btn2{ "..." }, btn3{ "close" }, btn4{ "..." };

    MyAudioProcessorEditor (MyAudioProcessor& p) : AudioProcessorEditor{ p }
    {
        container1.ignore = [&] { return container2.isVisible(); };
        container2.ignore = [ ] { return false; };

        btn1.onClick = [this]
        {
            container2.setVisible (true);
            container1.invalidateAccessibilityHandler();
            container2.grabKeyboardFocus();
        };

        btn3.onClick = [this]
        {
            container2.setVisible (false);
            container1.invalidateAccessibilityHandler();
            container1.grabKeyboardFocus();
        };

        container1.setTitle ("container 1");
        container2.setTitle ("container 2");
        container1.addAndMakeVisible (btn1);
        container1.addAndMakeVisible (btn2);
        container2.addAndMakeVisible (btn3);
        container2.addAndMakeVisible (btn4);
        addAndMakeVisible (container1);
        addChildComponent (container2);
        setSize (400, 400);
    }

    void resized() override
    {
        int w{ getWidth() / 2 }, h{ getHeight() / 2 };

        container1.setBounds (getLocalBounds());
        container2.setBounds (getLocalBounds());
        btn1.setBounds (0, 0, w, h);
        btn2.setBounds (w, 0, w, h);
        btn3.setBounds (0, h, w, h);
        btn4.setBounds (w, h, w, h);
    }

    JUCE_DECLARE_NON_COPYABLE_WITH_LEAK_DETECTOR (MyAudioProcessorEditor)
};

Clicking both “open” and “close” with the mouse makes the cursor get stuck on “close”. This doesn’t show the extent of the issue though -in more complex cases the tree gets really broken. It seems to be fixed by using UIA_LayoutInvalidatedEventId instead of UIA_StructureChangedEventId for elementCreated/Destroyed.

(edit) I’m not using this approach now -it’s easier to use enablement, and somehow it doesn’t need handler invalidation, so it doesn’t cause this issue. Anyway, I find the whole thing rather obscure. Between the two kinds of focus, the new getAllComponents() and accessibility roles, there are several ways to go about the same things that result in slight and confusing differences. I spent an inordinate amount of time getting this to work on a layout that was previously working and clear, and I’m not even sure why some things fail and others don’t. It’s particularly puzzling how the two focuses relate to each other: we can have different containers and traversers, and yet if one focus moves and the other doesn’t we have a problem.

Think of focusContainer and keyboardFocusContainer as two separate concerns - a component can be one of these, none of them, or both, depending on how you want it to behave.

A “focusable” component is one that is enabled and visible (visible also includes components which are obscured by others like the overlay in your first example as this gets complicated when things like AffineTransforms are involved) and a “keyboard focusable” component is a subset of this which has been marked as keyboard focusable with setWantsKeyboardFocus (true);.

The focusContainer flag is used when determining the accessibility element hierarchy that is exposed to a screen reader, so if component A has children B and C and is marked as a focusContainer then the screen reader client sees:

                                          A
                                         | |
                                         B C

where A is the “parent” of B and C in the hierarchy and B and C are siblings. This is used by Narrator and VoiceOver when navigating the UI with the navigation keys.

The keyboardFocusContainer flag is used internally by JUCE to determine which component will receive the keyboard focus when navigating with the tab key or clicking on components. In the above example if A is marked as a keyboardFocusContainer and B and C want the keyboard focus, then clicking on A will “trap” the keyboard focus in the container and tabbing will move between B and C.

We need these two separate focus mechanisms because there needs to be a way of controlling the accessibility hierarchy and focus order that is independent from keyboard focus as you will likely want to expose components to the screen reader client which should not be keyboard focusable, and their parent components shouldn’t interfere with the tab focus by trapping it. However, tabbing and moving keyboard focus should grab the accessibility focus - for example if you are focused on a JUCE window which contains a TextEditor and move keyboard focus to it with the tab key, the screen reader should also move its focus to the editor.

In both of your examples, I think you want to use FocusContainerType::focusContainer and not keyboardFocusContainer as you are trying to control the accessibility navigation.

4 Likes

That’s more or less what I had got. I need both containers, I’m trying to control both navigations. It was all smooth until I tried to get around this issue with hidden components. I understand the keyboard focus behaviour can’t be replicated because keyboard focus cycles around children but automation focus doesn’t, so when it reaches the start or the end it “falls” to the rest of the tree. Somehow we have to effectively hide the hidden components for automation when an overlay is shown. That’s where a number of options appear:

  • We could group the hidden components in a container and then
    • temporarily set this container to AccessibilityRole::ignored, invalidating the handler after each change. This is where the glitch happens: if the overlay is shown and closed using the mouse, the automation tree gets broken, as if handler invalidation were not reporting the structure change correctly. For complete weirdness, it doesn’t happen using the keyboard.
    • temporarily disable this container with setEnabled (false). Somehow this works without handler invalidation -I suppose the component is still in the automation tree, but it rejects focus.
  • We could use custom traversers to conditionally change the set of focusable components. This was the first thing I tried and it failed over and over. It’s not at all clear to me when a certain traverser will be created and queried. getAllComponents() and getDefaultComponent() both have a parent argument, as if they could be called for a different container than the one who created the traverser. When an overlay is visible, Narrator’s scan mode can get out of the plugin and back into it, which returns focus to the top component. I don’t know if I should handle everything in a single top component traverser, or make all traversers be able to deal with any container, or conditionally return a child container’s traverser from its parent… it’s a mess.

All these options can contradict each other and make the whole thing break. So I think there should be some, say, official word on how to deal with overlays, which are a common use case. If nothing is done, automation sees everything. None of the solutions I tried are exactly trivial. I’m using enablement now and it seems to work, but it required a change of layout, grouping the whole hidden layer in a component.

I’m not sure I totally understand the problem - if you set the visibility of the hidden components with setVisible (false) they will be ignored. This seems like the simplest and most logical solution and doesn’t require modifying focus containers or implementing custom focus traversal unless there is something I’m missing.

That’s a semitransparent overlay for overwrite confirmation. Automation can navigate the whole browser behind when the overlay is open. The browser should be visible, but inaccessible.

Anyway, using setEnabled() instead of setVisible() works, but as I said, it requires a change of layout. Previously, we could add an overlay directly to a parent component, as a sibling of everything behind:

browser
|    |       |     |
list buttons boxes overlay

Doing that now would require calling setEnabled (or setVisible, for that matter) on each single control, so now we need something like

container
|       |
overlay browser
        |    |       |
        list buttons boxes

For that case (overlays, deciding that some component should not be able to receive focus although they are visible because I know they are below some opaque component) I have patched my juce copy by adding a lamba ‘isComponentCurrentlyFocusableCallback’ in juce::FocusTraversers (and KeyboardFocusTraverser), that is called (if non null) in that loop inside FocusTraverser::findAllComponents:

for (auto* c : localComponents)
{
#if FOCUS_CALLBACK
    if (FocusTraverser::isComponentCurrentlyFocusableCallback() != nullptr &&
        FocusTraverser::isComponentCurrentlyFocusableCallback()(c) == false)
      continue;
#endif 
    components.push_back (c);
    if (! (c->*isFocusContainer)())
        findAllComponents (c, components, isFocusContainer);
}

and also , in juce_KeyboardFocusTraverser.cpp:

static bool isKeyboardFocusable (const Component* comp, const Component* container)
{
#if FOCUS_CALLBACK
    bool nope = (KeyboardFocusTraverser::isComponentCurrentlyFocusableCallback() != nullptr &&
                 KeyboardFocusTraverser::isComponentCurrentlyFocusableCallback()(const_cast<Component*>(comp)) == false);
    if (nope) return false;
#endif

    return comp->getWantsKeyboardFocus() && container->isParentOf (comp);
}

It seems to work pretty well