Android - UI Scaling Factor Bug

Android
1

I’ve been investigating Android use of UI Scale Factor, which doesn’t work properly for me.

General symptoms:

  • the UI scales as expected, but the app window doesn’t fill the screen and can be offset [expected behaviour: fill the screen as on iOS]
  • touch events are scaled correctly (but can be offset), but only work in an increasingly small area of window (the bottom and right areas become unresponsive as the scaling factor increases!) [expected behaviour: respond accurately in the full app display area]

Why I would like this fixed:

  • this affects the usability of my Android app for visually impaired users, who would like to be able use my app on Android with a scaled UI

Test methodology:

  • I’m changing the scaling factor from a UI component in my app, to track how the UI scales as I change the scaling factor in real-time
  • to toggle OpenGL / not Open GL, I change some code and restart my app

===

Summary of Findings

What actually happens, depends on whether or not OpenGL is enabled in the app:

With OpenGL enabled:

  • the app has an ugly black display area at the top, and displays anchored on the left, with the black display area above it; the height of this black area increases as the UI Scaling Factor is increased; the bottom part of the app is truncated by the height of the black area at the top
  • the app fills only part of the screen; the amount of space on top/right/bottom not filled by the app, increases as the UI Scaling Factor is increased above 1
  • touch events are interpreted as being at a lower position on the screen than where the user taps; so if you tap at x,y = 10,20; that might be interpreted by the app as happening at 10,120; where the Y offset is equal to the height of the black bar at the top
  • touch events are increasingly ignored in the X dimension as the scaling factor increases above 1; the ignored area on the right seems roughly equal to that of the missing UI area on the right
  • touch events are increasingly ignored in the Y dimension as the scaling factor increases above 1; the ignored area at the bottom seems roughly equal to that of the missing UI area at the bottom
  • other than that, touch events are scaled properly to the scaled UI

image
image1604×2572 190 KB

With OpenGL disabled:

  • the UI is anchored to the top left, but fills only part of the screen; the amount of space on right/bottom not filled by the app, increases as the UI Scaling Factor is increased above 1
  • touch events are increasingly ignored in the X dimension as the scaling factor increases above 1; the ignored area on the right seems roughly equal to that of the missing UI area on the right
  • touch events are increasingly ignored in the Y dimension as the scaling factor increases above 1; the ignored area at the bottom seems roughly equal to that of the missing UI area at the bottom
  • other than that, touch events are scaled properly to the scaled UI

image
image1604×2572 199 KB

See also this historic post:

Thanks! Pete

Android Pixel size - dpi - scalefactor
2

I’ve created a partial fix here for one of the issues: where touch events on the right/bottom of the app are ignored as Scaling Factor is increased:

modules/juce_gui_basics/native/juce_Windowing_android.cpp

    bool contains (Point<int> localPos, bool trueIfInAChildWindow) const override
    {
      // MARK - MPC - UI Scaling - don't ignore touch events at edges of scaled area (begin)
      if (trueIfInAChildWindow) {
        if (view.callBooleanMethod (ComponentPeerView.containsPoint,
                                    (float) localPos.x * scale,
                                    (float) localPos.y * scale))
        {
          // MPC - if the scaled position is in the ComponentPierView, then we have a hit
          return true;
        }
        return false;
      }

      if (isPositiveAndBelow (localPos.x, component.getWidth())
             && isPositiveAndBelow (localPos.y, component.getHeight())) {
        return true;
      }

      return false;

      /*
        return isPositiveAndBelow (localPos.x, component.getWidth())
            && isPositiveAndBelow (localPos.y, component.getHeight())
            && ((! trueIfInAChildWindow) || view.callBooleanMethod (ComponentPeerView.containsPoint,
                                                                    (float) localPos.x * scale,
                                                                    (float) localPos.y * scale));
       */
      // MARK - MPC - UI Scaling - don't ignore touch events at edges of scaled area (end)
    }
3

Here is the fix for the Window scaling, which is the second of the two issues.

I’ve tested the fix both with, and without, Open GL.

I hope you can merge these two fixes in ASAP!

    void setBounds (const Rectangle<int>& userRect, bool isNowFullScreen) override
    {
        // MARK: MPC fix for scaling factor (begin)
        float masterScale = Desktop::getInstance().getGlobalScaleFactor();
        // float useScale = scale;
        float useScale = scale * masterScale;
        //auto bounds = (userRect.toFloat() * scale).toNearestInt();
        auto bounds = (userRect.toFloat() * useScale).toNearestInt();
        // MARK: MPC fix for scaling factor (end)

        if (MessageManager::getInstance()->isThisTheMessageThread())
        {
            fullScreen = isNowFullScreen;

            view.callVoidMethod (AndroidView.layout,
                                 bounds.getX(), bounds.getY(), bounds.getRight(), bounds.getBottom());

            if (viewGroup != nullptr && viewGroupIsWindow)
            {
                auto* env = getEnv();

                LocalRef<jobject> windowLayoutParams (env->NewObject (AndroidWindowManagerLayoutParams, AndroidWindowManagerLayoutParams.create,
                                                                      bounds.getWidth(), bounds.getHeight(), bounds.getX(), bounds.getY(),
                                                                      TYPE_APPLICATION, FLAG_NOT_TOUCH_MODAL | FLAG_LAYOUT_IN_SCREEN | FLAG_LAYOUT_NO_LIMITS,
                                                                      component.isOpaque() ? PIXEL_FORMAT_OPAQUE : PIXEL_FORMAT_TRANSPARENT));

                env->SetIntField (windowLayoutParams.get(), AndroidWindowManagerLayoutParams.gravity, GRAVITY_LEFT | GRAVITY_TOP);
                env->CallVoidMethod (viewGroup.get(), AndroidViewManager.updateViewLayout, view.get(), windowLayoutParams.get());
            }
        }
        else
        {
            GlobalRef localView (view);

            MessageManager::callAsync ([localView, bounds]
            {
                localView.callVoidMethod (AndroidView.layout,
                                          bounds.getX(), bounds.getY(), bounds.getRight(), bounds.getBottom());
            });
        }
    }

Hoping this helps folks!

Best wishes,

Pete

4

Hi there, have you tested against the develop tip? There have been quite a few changes recently to juce_Windowing_android.cpp and to that function in particular.

5

Hi!

No, I’ve not.

TBH, I’ve got so many fixes applied to my own JUCE branch, that I won’t risk/attempt to merge a JUCE update unless absolutely necessary.

Best wishes, Pete

2 Likes
6

Hi @aamf

I today created a quick test project using JUCE 8, and found that:

  1. it looks like my suggested change to setBounds is only required for JUCE 7.
  2. however, my suggested change for “contains” does seem to be required as well as JUCE 8.

I hope that helps.

Best wishes, Pete

1 Like
7

Ah great, thanks!

1 Like
8

Thanks for the issue report. We’ve added fixes for view scaling and touch input scaling when a custom global scale factor is applied:

Please try out the changes and let us know if you run into any new issues.

1 Like