Bounds on mobile devices

Hi there!

First of all I’d like to thank the JUCE team for getting the Android windowing, insets, etc. working properly. One would expect this to be almost trivial but the amount of tedious work needed on Android for this stuff is beyond all measure. So many thanks once again!

That being said, there are a few things which are still not working perfectly, both on Android as on iOS.

One issue is being able to toggle freely between kiok mode and normal mode (that is, showing and hiding the status and navigation bars). The problem is that after doing so, rotating the device causes the wrong bounds to be set. One would expect that by setting the top level component as fullscreen the app’s bounds would be handled automatically, but this is not the case.

Below is a very simple test app which includes a simple workaround for this problem. I have tested it on different Android and iOS devices, using split mode or not, and the workaround seems to work fine. But it is clearly more of a hack, so perhaps the JUCE team could look into this and have the bounds set properly without any special intervention.

#include <JuceHeader.h>

class MainComponent  : public juce::Component
{
public:
    MainComponent() {}
    
    void resized() override { repaint(); }
    
    void paint (juce::Graphics& g) override
    {
       #if JUCE_IOS || JUCE_ANDROID
        const auto* display = juce::Desktop::getInstance().getDisplays().getDisplayForRect (getScreenBounds());
        auto safeBounds = display->safeAreaInsets.subtractedFrom (display->keyboardInsets.subtractedFrom (getLocalBounds()));
       #else
        auto safeBounds = getLocalBounds();
       #endif
    
        g.fillAll (isDarkStyle() ? juce::Colours::darkblue : juce::Colours::lightblue);
        
        g.setColour (isDarkStyle() ? juce::Colours::darkgreen : juce::Colours::lightgreen);
        g.fillRect (safeBounds);
        
        g.setColour (juce::Colours::red);
        g.drawRect (safeBounds.reduced (4), 3.0f);

        g.setColour (isDarkStyle() ? juce::Colours::white : juce::Colours::black);
        g.drawFittedText (getDisplayInfo(), safeBounds.reduced (10), juce::Justification::topLeft, 30);
    }
    
    juce::String getDisplayInfo()
    {
        auto* display = juce::Desktop::getInstance().getDisplays().getPrimaryDisplay();
        auto totalArea = display->totalArea.toString();
        auto userArea = display->userArea.toString();
        auto localBounds = getLocalBounds().toString();
        auto insets = display->safeAreaInsets;
        auto top = juce::String (insets.getTop());
        auto bottom = juce::String (insets.getBottom());
        auto left = juce::String (insets.getLeft());
        auto right = juce::String (insets.getRight());
        auto parentBounds = getTopLevelComponent()->getBounds().toString();
        auto peerNonFullscreen = getTopLevelComponent()->getPeer()->getNonFullScreenBounds().toString();
        auto kioskMode = isKioskMode() ? "on" : "off";
        auto appStyle = isDarkStyle() ? "dark" : "light";
        auto nl = juce::newLine;
        
        return juce::String ("totalArea: " + totalArea + nl +
                             "userArea: " + userArea + nl +
                             "localBounds: " + localBounds + nl +
                             "insets:" + " top " + top + " bottom " + bottom + " left " + left + " right " + right + nl +
                             "parentBounds: " + parentBounds + nl +
                             "peer non-fullscreen bounds: " + peerNonFullscreen + nl +
                             "kiosk mode: " + kioskMode + " (tap to change)" + nl +
                             "app style: " + appStyle + " (hold to change)" );
    }

    bool isKioskMode()
    {
        return juce::Desktop::getInstance().getKioskModeComponent() != nullptr;
    }
    
    void toggleKioskMode()
    {
        juce::Desktop::getInstance().setKioskModeComponent (isKioskMode() ? nullptr : getTopLevelComponent());
        
        // This ensures that disabling kiosk mode sets the correct bounds in all situations.
        // See the top level component's parentSizeChanged() function.
        getTopLevelComponent()->parentSizeChanged();
    }
        
    bool isDarkStyle()
    {
        return getTopLevelComponent()->getPeer()->getAppStyle() == juce::ComponentPeer::Style::dark;
    }
    
    void toggleAppStyle()
    {
        getTopLevelComponent()->getPeer()->setAppStyle (isDarkStyle() ? juce::ComponentPeer::Style::light 
                                                                      : juce::ComponentPeer::Style::dark);
    }
    
    void mouseUp (const juce::MouseEvent& e) override
    {
        if (e.mouseWasClicked() && e.getNumberOfClicks() == 1)
            toggleKioskMode();
        else if (e.mouseWasDraggedSinceMouseDown() && e.getDistanceFromDragStart() < 10)
            toggleAppStyle();
            
        repaint();
    }

    JUCE_DECLARE_NON_COPYABLE_WITH_LEAK_DETECTOR (MainComponent)
};

class NewProjectApplication  : public juce::JUCEApplication
{
public:
    NewProjectApplication() {}
    const juce::String getApplicationName() override            { return ProjectInfo::projectName; }
    const juce::String getApplicationVersion() override         { return ProjectInfo::versionString; }
    bool moreThanOneInstanceAllowed() override                  { return false; }
    void initialise (const juce::String& commandLine) override  { mainWindow.reset (new MainWindow (getApplicationName())); }
    void shutdown() override                                    { mainWindow = nullptr; }
    void systemRequestedQuit() override                         { quit(); }

    class MainWindow    : public juce::DocumentWindow
    {
    public:
        MainWindow (juce::String name)
            : DocumentWindow (name, juce::Colours::grey, DocumentWindow::allButtons)
        {
            setUsingNativeTitleBar (true);
            setContentOwned (new MainComponent(), false);

           #if JUCE_IOS || JUCE_ANDROID
            setFullScreen (true);
           #else
            centreWithSize (600, 400);
           #endif

            setResizable (true, true);
            setVisible (true);
        }
        
       #if JUCE_IOS || JUCE_ANDROID
        void parentSizeChanged() override
        {
            DocumentWindow::parentSizeChanged();
            
            // Without this, setting kiosk mode on and off and then rotating the device
            // leads to incorrect bounds being set, both on Android and on iOS.
            // This hack assumes that the app is fullscreen (checking isFullScreen()
            // here doen't work). Also, a small time delay is necessary.
            juce::Timer::callAfterDelay (50, [this]
            {
                const auto* display = juce::Desktop::getInstance().getDisplays().getDisplayForRect (getScreenBounds());
                setBounds (display->userArea);
                
                // It can be that the bounds have not changed but the safe area insets have,
                // so it's best to call resized() anyways.
                getContentComponent()->resized();
            });
        }
       #endif
        
        MainComponent& getMainComponent() { return *dynamic_cast<MainComponent*> (getContentComponent()); }
        void closeButtonPressed() override { JUCEApplication::getInstance()->systemRequestedQuit(); }
        ~MainWindow() { clearContentComponent(); }
    
        JUCE_DECLARE_NON_COPYABLE_WITH_LEAK_DETECTOR (MainWindow)
    };

private:
    std::unique_ptr<MainWindow> mainWindow;
};

START_JUCE_APPLICATION (NewProjectApplication)

/*
NOTE:
There is a difference between iPhones and other devices regarding the safe area insets.
On iPhones, if the status bar is displayed on landscape mode (i.e. when kiosk mode is
disabled), the reported top safe area inset is 0. This is intentional as Apple consider the
status bar as being non-intrusive in that scenario, and the status bar usually gets hidden
when entering landscape mode (note that when using JUCE you have to do so manually!). On iPads
and on Android devices, on the other hand, when the status bar is displayed on landscape mode,
the top area inset *does* reflect the height of the status bar.
*/

With regards to Android, two major things are still missing:

One is opening an app as a pop-up view. This is a feature mostly found in Samsung devices but since they are very popular it should be implemented. Currently, if a user opens a JUCE app as pop-up view, the app is placed on the centre of the screen with no way to move or resize it, so the user is forced to close it.

The other missing feature is being able to change the app style between dark and light modes. It has been implemented for iOS but not for Android. I mention this here because toggling kiosk mode affects the flags used to set the app style, so both need to be addressed together. I got it to work by adding this to juce_Windowing_android.cpp:

void appStyleChanged() override
{
    LocalRef<jobject> activity (getMainActivity());
    
    if (activity == nullptr)
        return;
    
    auto* env = getEnv();
    LocalRef<jobject> mainWindow (env->CallObjectMethod (activity.get(), AndroidActivity.getWindow));
    jclass windowClass = env->FindClass ("android/view/Window");
    jclass viewClass = env->FindClass ("android/view/View");
                                                                                               
    if (getAndroidSDKVersion() >= 30)
    {
        jmethodID getInsetsControllerMethod = env->GetMethodID (windowClass, "getInsetsController", "()Landroid/view/WindowInsetsController;");
        jobject insetsController = env->CallObjectMethod (mainWindow.get(), getInsetsControllerMethod);
        
        jclass windowInsetsControllerClass = env->FindClass ("android/view/WindowInsetsController");
        jmethodID setSystemBarsAppearanceMethod = env->GetMethodID (windowInsetsControllerClass, "setSystemBarsAppearance", "(II)V");
        
        constexpr int APPEARANCE_LIGHT_STATUS_BARS = 1 << 3;
        constexpr int APPEARANCE_LIGHT_NAVIGATION_BARS = 1 << 4;
        constexpr int LIGHT_MODE_FLAGS = APPEARANCE_LIGHT_STATUS_BARS | APPEARANCE_LIGHT_NAVIGATION_BARS;
        
        env->CallVoidMethod (insetsController, setSystemBarsAppearanceMethod,
                             style == Style::light ? LIGHT_MODE_FLAGS : 0, LIGHT_MODE_FLAGS);
    }
    else // SDK >= 26
    {
        constexpr int WHITE = 0xffffffff;
        constexpr int BLACK = 0xff000000;
    
        jmethodID setStatusBarColorMethod = env->GetMethodID (windowClass, "setStatusBarColor", "(I)V");
        jmethodID setNavigationBarColorMethod = env->GetMethodID (windowClass, "setNavigationBarColor", "(I)V");
        
        auto color = (style == Style::light && getAndroidSDKVersion() >= 27) ? WHITE : BLACK;
        
        env->CallVoidMethod (mainWindow.get(), setStatusBarColorMethod, color);
        env->CallVoidMethod (mainWindow.get(), setNavigationBarColorMethod, color);
        
        jmethodID getDecorViewMethod = env->GetMethodID (windowClass, "getDecorView", "()Landroid/view/View;");
        auto decorView = env->CallObjectMethod (mainWindow.get(), getDecorViewMethod);
        
        jmethodID setBackgroundColorMethod = env->GetMethodID (viewClass, "setBackgroundColor", "(I)V");
        auto rootView = env->CallObjectMethod (decorView, AndroidView.getRootView);

        env->CallVoidMethod (rootView, setBackgroundColorMethod, style == Style::light ? WHITE : BLACK);
        
        jmethodID getSystemUiVisibilityMethod = env->GetMethodID (viewClass, "getSystemUiVisibility", "()I");
        auto flags = env->CallIntMethod (decorView, getSystemUiVisibilityMethod);
        
        constexpr int SYSTEM_UI_FLAG_LIGHT_NAVIGATION_BAR = 16;
        constexpr int SYSTEM_UI_FLAG_LIGHT_STATUS_BAR = 8192;
        constexpr int LIGHT_MODE_FLAGS = SYSTEM_UI_FLAG_LIGHT_STATUS_BAR | SYSTEM_UI_FLAG_LIGHT_NAVIGATION_BAR;
        
        if (style == Style::light)
            flags |= LIGHT_MODE_FLAGS;
        else
            flags &= ~LIGHT_MODE_FLAGS;

        jmethodID setSystemUiVisibilityMethod = env->GetMethodID (viewClass, "setSystemUiVisibility", "(I)V");
        env->CallVoidMethod (decorView, setSystemUiVisibilityMethod, flags);
    }
    
    setNavBarsHidden (navBarsHidden);
}

Hope this helps!

Thanks for the kind words! Getting this stuff working is surprisingly difficult, and it’s taken multiple weeks of dev time so far…

I think what’s happening here is that exiting kiosk mode attempts to knock the component out of full-screen mode too, and restore its previous (non-full-screen) bounds. I’m not sure whether we should change this behaviour, as this would affect existing apps. Changing it only on mobile platforms might make sense, but I’m not sure one way or the other.

In initial testing, the demo code seems to behave correctly when removing the ‘workaround’ code, and adding the following at the end of toggleKioskMode():

if (auto* peer = getPeer())
    peer->setFullScreen (true);

Please could you give that a go and see whether you can break it? If it works for you, that feels preferable to making internal changes.

This is going to be awkward to test, as I don’t have any Samsung/Android devices here. Your example seems to behave correctly in a ‘Desktop’ Android emulator, which allows apps to be placed in resizable windows with a titlebar. This sounds pretty similar to the Samsung feature, but maybe the Samsung feature has other requirements.

Have you tried out popup windows with kiosk mode initially disabled? I wonder whether kiosk mode is also disabling the controls to move/resize these windows. Additionally, are you aware of a way to enable a similar feature in a standard emulator image?

Yep, we should definitely add that. Thanks for the example code!

1 Like

Ah brilliant, that works perfectly! I tested on all my devices and wasn’t able to break it, everything works as it should. (I could swear I had tried something like that though…) Yes, I don’t think it’s necessary to make internal changes, it would be enough to mention this in the documentation.

Yes, I have, but it makes no difference. The Samsung feature definitely behaves differently to the “normal” Android resizable windows (which are working fine) and has other requirements, I’m afraid.

For example, I can detect when the app is in “pop-up mode” by checking if the user area’s width and height are smaller than the screen bounds. If I then do something like this:

auto bounds = display->userArea;
bounds.removeFromTop (40); 
setBounds (bounds);

then the content is placed under the control bars as one would expect, but for some reason I am not able to resize or move the window.

If I instead set the document window to not visible, then resizing and moving the window are possible.

Certainly! I’m not aware of a way to enable a similar feature in a standard emulator. This seems to be very Samsung-specific.

I hope it helps you save some time. Also, due to the recent changes I had to modify setSystemUiVisibilityCompat within ComponentPeerView.java (and of course the corresponding call and JNI definition):

public void setSystemUiVisibilityCompat (Window window, boolean visible, boolean lightMode)
{
    if (30 <= Build.VERSION.SDK_INT)
    {
        WindowInsetsController controller = getWindowInsetsController();

        if (controller != null)
        {
            if (visible)
            {
                controller.show (WindowInsets.Type.systemBars());
                controller.setSystemBarsBehavior (31 <= Build.VERSION.SDK_INT ? BEHAVIOR_DEFAULT
                                                                              : BEHAVIOR_SHOW_BARS_BY_SWIPE);
            }
            else
            {
                controller.hide (WindowInsets.Type.systemBars());
                controller.setSystemBarsBehavior (BEHAVIOR_SHOW_TRANSIENT_BARS_BY_SWIPE);
            }

            if (lightMode)
            {
                controller.setSystemBarsAppearance (WindowInsetsController.APPEARANCE_LIGHT_STATUS_BARS |
                                                    WindowInsetsController.APPEARANCE_LIGHT_NAVIGATION_BARS,
                                                    WindowInsetsController.APPEARANCE_LIGHT_STATUS_BARS |
                                                    WindowInsetsController.APPEARANCE_LIGHT_NAVIGATION_BARS);
            }
            else
            {
                controller.setSystemBarsAppearance (0,
                                                    WindowInsetsController.APPEARANCE_LIGHT_STATUS_BARS |
                                                    WindowInsetsController.APPEARANCE_LIGHT_NAVIGATION_BARS);
            }

            return;
        }
    }

    if (window == null)
        return;

    // Displays::findDisplays queries the DecorView to determine the
    // most recently-requested visibility state of the system UI.
    // As we're creating new top-level views via WindowManager,
    // updating only the DecorView isn't sufficient to hide the global
    // system UI; we also need to update the view that was added to
    // the WindowManager.
    ArrayList<View> views = new ArrayList<>();
    views.add (window.getDecorView());
    views.add (this);

    for (View view : views)
    {
        final int prevFlags = view.getSystemUiVisibility();
        final int fullScreenFlags = SYSTEM_UI_FLAG_HIDE_NAVIGATION
                                  | SYSTEM_UI_FLAG_FULLSCREEN
                                  | SYSTEM_UI_FLAG_IMMERSIVE_STICKY;
        int newFlags = visible ? (prevFlags & ~fullScreenFlags)
                               : (prevFlags |  fullScreenFlags);
                                     
        final int lightModeFlags = SYSTEM_UI_FLAG_LIGHT_NAVIGATION_BAR
                                 | SYSTEM_UI_FLAG_LIGHT_STATUS_BAR;
        newFlags = lightMode ? (newFlags | lightModeFlags)
                             : (newFlags &~ lightModeFlags);

        view.setSystemUiVisibility (newFlags);
    }
}

Hi there (and thanks again for looking into this)!

Since my Samsung device is limited to Android 13 (it’s a Galaxy A22 5G), I’ve been testing the “Open in PopUp View” function on different systems using both Android Device Streaming and Samsung’s Remote Test Lab (The latter works very well if you choose devices located in Europe. You can also find devices running older APIs).

I’ve discovered that the problem affects devices up to Android 13. Android 14 and higher work fine, with the only minor issue being that the corners are not rounded.

Android 13 and lower:

Android 14 and higher:

We’ve pushed a change to allow setting the style of the status and navigation bars:

I’ve been investigating the windowing issue over the last few days. Hopefully I’ll be able to provide a patch for testing before too long.

Actually, I think it’d be worth testing this patch, so that we know whether it resolves the OneUI windowing issue. Please could you try it out and let me know how you get on?

I’m still seeing one rare, intermittent issue where the window insets get applied twice after entering split-view mode on a phone. It only happens rarely, and I haven’t worked out what causes it yet. Other than that, the new pproach seems to work. I’d appreciate if you could test the patch and let me know if you spot any regressions or new issues.

android-windowing.patch (154.8 KB)

The OneUI windowing is working perfectly, I see no problems whatsoever. Also the reporting of the insets seems to work reliably. This is truly wonderful!

I can see the following issue on Samsung devices though (“normal” Android works fine): If navigation buttons are used instead of swipe gestures AND the system is set to light mode (both the default on Samsung devices), then, when the app is set to dark mode, the navigation buttons are drawn incorrectly (dark elements on a dark background). And on API 35 this even happens regardless of whether the system is set to dark or light mode. If swipe gestures are used, on the other hand, then the “swipe indicator bar” is always displayed correctly.

The navigation buttons are displayed correctly on my fork. I really can’t recall how exactly I got it to work though. I have been investigating your version but have not been able to find the culprit. I can give it another go after the weekend. Interestingly, when one enables kiosk mode, the navigation buttons are briefly drawn correctly as they animate out of the screen. Hmm…

I got it to work by calling window.setNavigationBarColor (0xff000000); within setSystemUiVisibilityCompat.

1 Like

Thanks for all your feedback on these issues. We’ve now merged the fix for windowed mode:

…and for the navigation bar icon colour:

Please let us know if you’re still seeing any issues after updating to the latest develop branch.

This is brilliant, thank you so much for your hard work on this… and patience!

I have been able to find a few minor issues:

If an OpenGL context is attached, turning off kiosk mode causes the screen to go black briefly. This ranges from about half a second to a few milliseconds, but it happens on all APIs and devices I have tested.

And on API <= 28, calling setAppStyle() from a thread other than the message thread causes a crash.

Finally (this is not really an issue but I though I should mention it) I don’t think it is entirely obvious that the safe area insets can change without resized() being called. Perhaps it would be good to mention it somewhere.

Apart from these very minor issues everything seems to be working pefrectly. Yay!

1 Like

Found another issue: if the device is using button navigation and the app is not in kiosk mode, displaying the Buetooth MIDI pairing dialogue corrupts the safe are insets and the status and navigation bar colours. This can be observed with the DemoRunner by running it not on kiosk mode, displaying the pairing dialogue, closing it, and then rotating the device. Tested on a Samsung A22 (API 33) and a Pixel 4A (API 34).

1 Like