JUCE Accessibility on `develop`

Thanks, the fix for this requires some fairly heavy refactoring of how macOS handles offscreen rows for tree and lists so it’ll be out sometime next week.

VO+shift+down will "enter’ the menu window and you can then navigate with VO+arrow keys.

Invoking the “show menu” command with VO+shift+m will show the submenu, or navigating with the arrow keys without VO modifiers when the menu has keyboard focus.

These are both good suggestions, I’ll look at adding them.

Thanks, I’ll look into the Narrator text interface issues next week.

The container hierarchy works slightly differently on Windows to macOS and will present UI elements as a flat tree, you grab the tab focus in a component by setting its focus container type to FocusContainerType::keyboardFocusContainer and it will behave as before.

I think that’s what I’m doing. Here is a minimal example.

struct Overlay : juce::Component
{
    Overlay()
    {
        setFocusContainerType (FocusContainerType::keyboardFocusContainer);
        exit.onClick = [this] { setVisible (false); };
        addAndMakeVisible (exit);
    }

    void paint (juce::Graphics& g) override
    {
        g.fillAll (juce::Colours::lightgrey);
    }

    void resized() override
    {
        exit.setBounds (getWidth() / 2 + 20, 20, getWidth() / 2 - 40, getHeight() - 40);
    }

private:
    juce::TextButton exit{ "exit" };
};

struct MyAudioProcessorEditor : juce::AudioProcessorEditor
{
    MyAudioProcessorEditor (MyAudioProcessor& p) : AudioProcessorEditor{ p }
    {
        enter.onClick = [&]
        {
            overlay.setVisible (true);
            overlay.grabKeyboardFocus();
        };
        addAndMakeVisible (enter);
        addChildComponent (overlay);
        setSize (600, 400);
    }

    void paint (juce::Graphics& g) override
    {
        g.fillAll (juce::Colours::darkgrey);
    }

    void resized() override
    {
        enter.setBounds (20, 20, getWidth() / 2 - 40, getHeight() - 40);
        overlay.setBounds (getLocalBounds());
    }

private:
    Overlay overlay;
    juce::TextButton enter{ "enter" };
    JUCE_DECLARE_NON_COPYABLE_WITH_LEAK_DETECTOR (MyAudioProcessorEditor)
};

tab navigation is ok, but when Narrator is in scan mode (Caps+space), you can navigate with arrow up/down and reach controls outside the focus container -in this case, the “enter” button when it’s not visible, behind the overlay.

(edit) I tried making all the overlays modal (which wasn’t necessary before) -it’s the same. UI automation can interact with components behind modal components.

A tester has reported to me a crash in juce_mac_Accessibility.mm line 67

Holder element ([cls.createInstance() init]);

It happens during application startup. He is on macOS 10.9 so that’s probably the cause of the crash.

I was trying to approach the navigation issue with a variable (ignored/group) role and handler invalidation, and I ran into this glitch: when Narrator is active, and the top layer is both shown and hidden using the mouse, automation focus gets stuck on a hidden component. This would be a simple case:

struct Container : juce::Component
{
    Container()
    {
        setFocusContainerType (FocusContainerType::keyboardFocusContainer);
    }

    std::function<bool()> ignore;

    std::unique_ptr<juce::AccessibilityHandler> createAccessibilityHandler() override
    {
        auto role{ ignore() ? juce::AccessibilityRole::ignored : juce::AccessibilityRole::group };
        return std::make_unique<juce::AccessibilityHandler> (*this, role);
    }
};

struct MyAudioProcessorEditor : juce::AudioProcessorEditor
{
    Container container1, container2;
    juce::TextButton btn1{ "open" }, btn2{ "..." }, btn3{ "close" }, btn4{ "..." };

    MyAudioProcessorEditor (MyAudioProcessor& p) : AudioProcessorEditor{ p }
    {
        container1.ignore = [&] { return container2.isVisible(); };
        container2.ignore = [ ] { return false; };

        btn1.onClick = [this]
        {
            container2.setVisible (true);
            container1.invalidateAccessibilityHandler();
            container2.grabKeyboardFocus();
        };

        btn3.onClick = [this]
        {
            container2.setVisible (false);
            container1.invalidateAccessibilityHandler();
            container1.grabKeyboardFocus();
        };

        container1.setTitle ("container 1");
        container2.setTitle ("container 2");
        container1.addAndMakeVisible (btn1);
        container1.addAndMakeVisible (btn2);
        container2.addAndMakeVisible (btn3);
        container2.addAndMakeVisible (btn4);
        addAndMakeVisible (container1);
        addChildComponent (container2);
        setSize (400, 400);
    }

    void resized() override
    {
        int w{ getWidth() / 2 }, h{ getHeight() / 2 };

        container1.setBounds (getLocalBounds());
        container2.setBounds (getLocalBounds());
        btn1.setBounds (0, 0, w, h);
        btn2.setBounds (w, 0, w, h);
        btn3.setBounds (0, h, w, h);
        btn4.setBounds (w, h, w, h);
    }

    JUCE_DECLARE_NON_COPYABLE_WITH_LEAK_DETECTOR (MyAudioProcessorEditor)
};

Clicking both “open” and “close” with the mouse makes the cursor get stuck on “close”. This doesn’t show the extent of the issue though -in more complex cases the tree gets really broken. It seems to be fixed by using UIA_LayoutInvalidatedEventId instead of UIA_StructureChangedEventId for elementCreated/Destroyed.

(edit) I’m not using this approach now -it’s easier to use enablement, and somehow it doesn’t need handler invalidation, so it doesn’t cause this issue. Anyway, I find the whole thing rather obscure. Between the two kinds of focus, the new getAllComponents() and accessibility roles, there are several ways to go about the same things that result in slight and confusing differences. I spent an inordinate amount of time getting this to work on a layout that was previously working and clear, and I’m not even sure why some things fail and others don’t. It’s particularly puzzling how the two focuses relate to each other: we can have different containers and traversers, and yet if one focus moves and the other doesn’t we have a problem.

Think of focusContainer and keyboardFocusContainer as two separate concerns - a component can be one of these, none of them, or both, depending on how you want it to behave.

A “focusable” component is one that is enabled and visible (visible also includes components which are obscured by others like the overlay in your first example as this gets complicated when things like AffineTransforms are involved) and a “keyboard focusable” component is a subset of this which has been marked as keyboard focusable with setWantsKeyboardFocus (true);.

The focusContainer flag is used when determining the accessibility element hierarchy that is exposed to a screen reader, so if component A has children B and C and is marked as a focusContainer then the screen reader client sees:

                                          A
                                         | |
                                         B C

where A is the “parent” of B and C in the hierarchy and B and C are siblings. This is used by Narrator and VoiceOver when navigating the UI with the navigation keys.

The keyboardFocusContainer flag is used internally by JUCE to determine which component will receive the keyboard focus when navigating with the tab key or clicking on components. In the above example if A is marked as a keyboardFocusContainer and B and C want the keyboard focus, then clicking on A will “trap” the keyboard focus in the container and tabbing will move between B and C.

We need these two separate focus mechanisms because there needs to be a way of controlling the accessibility hierarchy and focus order that is independent from keyboard focus as you will likely want to expose components to the screen reader client which should not be keyboard focusable, and their parent components shouldn’t interfere with the tab focus by trapping it. However, tabbing and moving keyboard focus should grab the accessibility focus - for example if you are focused on a JUCE window which contains a TextEditor and move keyboard focus to it with the tab key, the screen reader should also move its focus to the editor.

In both of your examples, I think you want to use FocusContainerType::focusContainer and not keyboardFocusContainer as you are trying to control the accessibility navigation.

4 Likes

That’s more or less what I had got. I need both containers, I’m trying to control both navigations. It was all smooth until I tried to get around this issue with hidden components. I understand the keyboard focus behaviour can’t be replicated because keyboard focus cycles around children but automation focus doesn’t, so when it reaches the start or the end it “falls” to the rest of the tree. Somehow we have to effectively hide the hidden components for automation when an overlay is shown. That’s where a number of options appear:

  • We could group the hidden components in a container and then
    • temporarily set this container to AccessibilityRole::ignored, invalidating the handler after each change. This is where the glitch happens: if the overlay is shown and closed using the mouse, the automation tree gets broken, as if handler invalidation were not reporting the structure change correctly. For complete weirdness, it doesn’t happen using the keyboard.
    • temporarily disable this container with setEnabled (false). Somehow this works without handler invalidation -I suppose the component is still in the automation tree, but it rejects focus.
  • We could use custom traversers to conditionally change the set of focusable components. This was the first thing I tried and it failed over and over. It’s not at all clear to me when a certain traverser will be created and queried. getAllComponents() and getDefaultComponent() both have a parent argument, as if they could be called for a different container than the one who created the traverser. When an overlay is visible, Narrator’s scan mode can get out of the plugin and back into it, which returns focus to the top component. I don’t know if I should handle everything in a single top component traverser, or make all traversers be able to deal with any container, or conditionally return a child container’s traverser from its parent… it’s a mess.

All these options can contradict each other and make the whole thing break. So I think there should be some, say, official word on how to deal with overlays, which are a common use case. If nothing is done, automation sees everything. None of the solutions I tried are exactly trivial. I’m using enablement now and it seems to work, but it required a change of layout, grouping the whole hidden layer in a component.

I’m not sure I totally understand the problem - if you set the visibility of the hidden components with setVisible (false) they will be ignored. This seems like the simplest and most logical solution and doesn’t require modifying focus containers or implementing custom focus traversal unless there is something I’m missing.

That’s a semitransparent overlay for overwrite confirmation. Automation can navigate the whole browser behind when the overlay is open. The browser should be visible, but inaccessible.

Anyway, using setEnabled() instead of setVisible() works, but as I said, it requires a change of layout. Previously, we could add an overlay directly to a parent component, as a sibling of everything behind:

browser
|    |       |     |
list buttons boxes overlay

Doing that now would require calling setEnabled (or setVisible, for that matter) on each single control, so now we need something like

container
|       |
overlay browser
        |    |       |
        list buttons boxes

For that case (overlays, deciding that some component should not be able to receive focus although they are visible because I know they are below some opaque component) I have patched my juce copy by adding a lamba ‘isComponentCurrentlyFocusableCallback’ in juce::FocusTraversers (and KeyboardFocusTraverser), that is called (if non null) in that loop inside FocusTraverser::findAllComponents:

for (auto* c : localComponents)
{
#if FOCUS_CALLBACK
    if (FocusTraverser::isComponentCurrentlyFocusableCallback() != nullptr &&
        FocusTraverser::isComponentCurrentlyFocusableCallback()(c) == false)
      continue;
#endif 
    components.push_back (c);
    if (! (c->*isFocusContainer)())
        findAllComponents (c, components, isFocusContainer);
}

and also , in juce_KeyboardFocusTraverser.cpp:

static bool isKeyboardFocusable (const Component* comp, const Component* container)
{
#if FOCUS_CALLBACK
    bool nope = (KeyboardFocusTraverser::isComponentCurrentlyFocusableCallback() != nullptr &&
                 KeyboardFocusTraverser::isComponentCurrentlyFocusableCallback()(const_cast<Component*>(comp)) == false);
    if (nope) return false;
#endif

    return comp->getWantsKeyboardFocus() && container->isParentOf (comp);
}

It seems to work pretty well

Hi

OK first: this is fantastic. Thank you. I’ve been porting Surge to JUCE and, by using develop tonight and adding a couple of lines here and there, was able to get proper names on all our sliders. Still loads work to do but this is really wonderful. Thank you for doing it.

A question about juce dev. Is there some ifdef guard I can use to protect code which requires develop branch vs a release? So something like

#if JUCE_VERSION_IS_DEVELOP
// do acc stuff here while I wait for 6.1 to ship but don’t update the other devs yet
#endif

so I can do an opt-in JUCE develop? I see develop at head defines JUCE_VERSION at 0x60008 (and presumably will be 0x610000 when 6.1 ships or something) but is there a way to distinguish dev from 6.0.8 in my C++? Sorry for the noob question but google didn’t help!

I have an issue with multiple notifications of new window being received when I open 1 new window.

Here is how to reproduce on DemoRunner: If I modify WindowsDemo.h this way:

void showAllWindows()
{
    closeAllWindows();

    showDocumentWindow (false);
    //showDocumentWindow (true);
    //showTransparentWindow();
    //showDialogWindow();
}


void showDocumentWindow (bool native)
{
  auto* dw = new DocumentWindow ("Document Window", getRandomBrightColour(), DocumentWindow::allButtons, false);
    windows.add (dw);
    (...)
    dw->setUsingNativeTitleBar (native);

    dw->addToDesktop(dw->getDesktopWindowStyleFlags());
    dw->setVisible (true);
}

So that only one windows is opened , and the addToDesktop arg of DocumentWindow contructor is false, and we call explicitely addToDesktop() before setVisible(true).

Then VoiceOver seems to receive 4 notifications of new window (instead of one). Sometimes it will read “DemoRun Demorunner has new windows” , but most often it will read “DemoRunner has new window DemoRunner has new window DemoRunner has new window DemoRunner has new window”

You’re not the first to wish there were, but unfortunately there’s no agreed upon solution so far. Some possible approaches have been discussed here, but (IIRC) without reaching a solution: JUCE_VERSION master and develop

1 Like

Ok thanks! I can bodge together a flag in my cmake file for use accessibility then and default it off. Appreciate the answer

This change in ListBox
listbox focus
breaks the behaviour at least for single selection: moving with arrow up/down doesn’t select anything with Narrator on.

The TextEditor issues should now be fixed on develop:

It is now also possible to provide read-only text > 1000 characters.

1 Like

With this change on develop, if the overlay is modal (using Component::enterModalState()) it’ll grab the focus and it won’t be possible to move the screen reader focus to blocked components. We’ve updated the Projucer’s log-in form to use this behaviour so you can see how it works there:

1 Like

This should be fixed on develop now. We rely on features only available on 10.10+ for the accessibility support on macOS so have disabled it for lower deployment targets:

1 Like

This has been fixed with the following commit:

1 Like

Thanks for reporting the regression, I’ve reverted that change here:

2 Likes