iOS Multitasking

RomanS · July 16, 2019, 4:15pm

Hi,

I am trying to implement Multitasking for iPads in iOS (two Apps side by side). To achieve this e.g. with the JUCE DemoRunner I just enabled all device orientations for iPad and unselected “Requires full screen”.

Now I am struggling with resizing my window to the space available. It seems Juce keeps setting the window to be the complete screen size of the device instead of just the part that is available for my app. This leads to a part of my app being concealed by the second app. Please see screenshot attached.

Digging into the iOS implementation it seems that
JuceUIViewController::viewWillTransitionToSize: (CGSize) size withTransitionCoordinator: (id) coordinator is not always called when I adjust the size of my window (but always when I change device orientation). And if it is called the size argument is ignored by Juce which instead uses Desktop::getInstance().getDisplays().getMainDisplay().userArea which in turn seems to hold the complete screen and not the portion that is available to my app.

Any ideas on how to fix this?

All the best,
Roman

PNG

peteatjuce · August 3, 2019, 4:23pm

Hi Roman, funnily enough I’m facing the same problem, and would love to have a solution.

Jules, any chance you can resolve this for us?

Pete

peteatjuce · August 4, 2019, 9:22am

I’ve been able to get this working for both cases:
a) when my app starts full screen, and another app is dragged-in via the task bar
b) where another app starts first and my app is dragged-up via the task bar.

My solution thus far is pretty simple:

in the .plist file:
UIRequiresFullScreen
in juce_ios_UIViewComponentPeer.mm:

// Declare two statics (yes, ugly, I know!)

CGSize GUseWindowSize;
static bool GbGotUseWindowSize = false;

//...

// Modify method:
- (void) viewWillTransitionToSize: (CGSize) size withTransitionCoordinator: (id<UIViewControllerTransitionCoordinator>) coordinator
{
    GUseWindowSize = size;
    GbGotUseWindowSize = true;

//...
// Modify constructor UIViewComponentPeer::UIViewComponentPeer
//...
{
    CGRect r = convertToCGRect (component.getBounds());

    if (GbGotUseWindowSize) {
        r.size = GUseWindowSize;
    }

//...
// Modify UIWindow constructor:
//...

        //window = [[JuceUIWindow alloc] initWithFrame: r];
        window = [[JuceUIWindow alloc] init]; // Multitasking change
        if (GbGotUseWindowSize == false) {
            GbGotUseWindowSize = true;
            GUseWindowSize = window.bounds.size;
            view.bounds.size = GUseWindowSize;
        }

// Disable flag setting:
        //window.autoresizesSubviews = NO;

Modify juce_ios_Windowing.mm

extern CGSize GUseWindowSize;
extern bool GbGotUseWindowSize;

void Desktop::Displays::findDisplays (float masterScale)
{
    JUCE_AUTORELEASEPOOL
    {
        UIScreen* s = [UIScreen mainScreen];

        auto theBounds = [s bounds];
        if (GbGotUseWindowSize) {
            theBounds.size = GUseWindowSize;
        }

        Display d;
        d.userArea = d.totalArea = UIViewComponentPeer::realScreenPosToRotated (convertToRectInt (theBounds)) / masterScale;

Of course, this is just a hack, maybe not all the changes are correct, and will no doubt need to be finessed.

That said: Jules & team - is there any chance you could merge-in a tidied-up version of the above into the code base?

Testing notes: to use multi-tasking on iPad, in case this helps!

start your app
drag-up enough to show the Task Bar
select icon in task bar, and drag to left/right of the current app; it’ll show a window where you app will appear
you can then drag the central splitter left/right to resize, or if you drag fully left/right both apps become full screen mode.
only some apps work of course - including the built-in Reminders app (very useful for testing on Simulator!) and Safari.

Thanks,

Pete

ed95 · August 5, 2019, 1:50pm

Thanks for looking into this and posting your solution. I’ll add this to our backlog and hopefully we’ll get around to looking at it soon.

peteatjuce · August 6, 2019, 10:11am

Thanks. Just tidied-up the code formatting a bit to make it clearer for you. As you can see, the changes are pretty simple! Pete

peteatjuce · January 7, 2021, 11:41am

Hi Folks, I’ve just started to look at transitioning the Wotja code base from JUCE 5 to JUCE 6.

I’ve put this off for a long time, as I’ve had to make a lot of fixes to JUCE over the past few years.

Anyhow, the most complex set of change is related to this issue. I see it still isn’t implemented in JUCE 6?

To verify this for yourself, is really really simple to prove.

in the .plist file: set UIRequiresFullScreen to NO
build and run your iPad app
Try using multi-tasking / split screen. Here is what I do (on Simulator)

a) where nothing yet running i.e. your app starts first

start your iPad app running
drag-up enough to show the Task Bar
start-up
select the Safari or Reminders app icon (both these apps support split screen) in task bar, and drag to left/right of the current app; it’ll show a window where you app will appear
you can then drag the central splitter left/right to resize, or if you drag fully left/right both apps become full screen mode.

b) where another app is running, and you want to see if your pre-deployed app starts-up properly in split screen mode:

start-up Safari (as that supports split screen mode) or Reminders app (ditto)
drag-up enough to show the Task Bar
select your app’s icon in task bar, and drag to left/right of the current app; it’ll show a window where you app will appear
you can then drag the central splitter left/right to resize, or if you drag fully left/right both apps become full screen mode.

I have had to change both juce_ios_UIViewComponentPeer.mm and juce_ios_Windowing.mm extensively to fix this again in JUCE6, but it is working well again! Do please contact me if you want my code, so you can patch it back in.

Best wishes as always,

Pete

reuk · January 15, 2021, 12:09pm

I’ve just pushed a fix which should enable support for iOS multitasking:

As before, you’ll need to make the following changes to your project to enable multitasking:

Set UIRequiresFullScreen to false in the app’s plist
Ensure that all screen orientations are enabled for the app

This patch seems to work well in the DemoRunner and AudioPluginHost - hopefully it will work well for your use-case too. Let us know if not, and I’ll take another look.

peteatjuce · January 19, 2021, 8:48am

Hi - thanks for this!

I’ve just been pushing-out a new Wotja release, but will hopefully try out your code and report back sometime later today; failing that, in just a day or two; merging-in changes from the main Juce distribution is always a bit tough, as I’ve had to patch Juce pretty heavily to work well cross platform!

Best wishes,

Pete

peteatjuce · January 22, 2021, 2:56pm

Hi! Sorry it has taken me a few days to get around to testing this out, but you know how it goes…

I’m very pleased to report that your changes worked perfectly for me. Great work - that’ll help a lot of your iOS users, I’m sure! I’m now using the Juce version, and have used that to replace my own changes. FWIW, I’ve received feedback from the Wotja user base, saying that split screen workflow with Wotja makes a huge difference for iPad users - so this is a great thing to have added.

On a separate, but related note, I have fixes for Juce in a few other important areas:

tvOS support (only affects a few files - juce_MidiDevices.h, juce_TargetPlatform.h, juce_FileChooser.h, juce_gui_basics.h, juce_io_UIViewComponentPeer.mm, juce_gui_extra.cpp)
pop-up menu positioning fix (especially for small Android devices, when large pop-up menu selected towards right or top or bottom of screen)
selection of “home” directory on Android (based around using getFilesDir in preference)
macOS Font Selection fixes
various others, too many to list really!

If you like me to sent you a zip of those changes, so you could at least consider merging them into the main distro, I’d be very happy to supply. Just DM me or mail me!

Best regards,

Pete

bastienc · February 12, 2021, 12:04pm

@reuk Could you consider adding the option for CMAKE too?

github.com

juce-framework/JUCE/blob/master/docs/CMake API.md#juce_add_target

# The JUCE CMake API

## System Requirements

- Console and GUI projects require CMake 3.12 or higher.
- Plugin projects require CMake 3.15 or higher.
- All iOS targets require CMake 3.14 or higher (3.15 or higher for plugins targeting iOS).
- Android targets are not currently supported.
- WebView2 on Windows via JUCE_USE_WIN_WEBVIEW2 flag in juce_gui_extra is not currently supported.

Most system package managers have packages for CMake, but we recommend using the most recent release
from https://cmake.org/download. You should always use a CMake that's newer than your build
toolchain, so that CMake can identify your build tools and understand how to invoke them.

In addition to CMake you'll need a build toolchain for your platform, such as Xcode or MSVC.

## Getting Started

### Using `add_subdirectory`

This file has been truncated. show original

reuk · February 12, 2021, 12:19pm

Which option specifically? Multitasking can be enabled in CMake by setting REQUIRES_FULL_SCREEN to FALSE, and by setting the IPHONE_SCREEN_ORIENTATIONS and IPAD_SCREEN_ORIENTATIONS to all possible orientations. The JUCE DemoRunner should have multitasking enabled, so you can check its CMakeLists:

github.com

juce-framework/JUCE/blob/develop/examples/DemoRunner/CMakeLists.txt#L34


juce_add_gui_app(DemoRunner
    BUNDLE_ID                       com.juce.demorunner
    ICON_BIG                        ${CMAKE_CURRENT_SOURCE_DIR}/Source/JUCEAppIcon.png
    NEEDS_CURL                      TRUE
    NEEDS_WEB_BROWSER               TRUE
    MICROPHONE_PERMISSION_ENABLED   TRUE
    CAMERA_PERMISSION_ENABLED       TRUE
    BLUETOOTH_PERMISSION_ENABLED    TRUE
    FILE_SHARING_ENABLED            TRUE
    DOCUMENT_BROWSER_ENABLED        TRUE
    REQUIRES_FULL_SCREEN            FALSE
    IPAD_SCREEN_ORIENTATIONS        UIInterfaceOrientationPortrait
                                    UIInterfaceOrientationPortraitUpsideDown
                                    UIInterfaceOrientationLandscapeLeft
                                    UIInterfaceOrientationLandscapeRight
    IPHONE_SCREEN_ORIENTATIONS      UIInterfaceOrientationPortrait
                                    UIInterfaceOrientationPortraitUpsideDown
                                    UIInterfaceOrientationLandscapeLeft
                                    UIInterfaceOrientationLandscapeRight)
juce_generate_juce_header(DemoRunner)

bastienc · February 16, 2021, 6:26pm

I did not have the option REQUIRES_FULL_SCREEN set, setting to TRUE still generates this following issue while validating the ipa and the info is not set in the plist. If I manually set the option within PLIST_TO_MERGE there is no validating issue anymore. This might be a regression. Can you confirm the plist entry is properly generated on your side ? Thanks



2021-02-16 19:05:08.107 altool[14798:356148] *** Error: Unable to validate archive '../Output/XXXXXXX.ipa'.
2021-02-16 19:05:08.107 altool[14798:356148] *** Error: code 1091 (App Store operation failed. Invalid Bundle. iPad Multitasking support requires these orientations: 'UIInterfaceOrientationPortrait,UIInterfaceOrientationPortraitUpsideDown,UIInterfaceOrientationLandscapeLeft,UIInterfaceOrientationLandscapeRight'. Found 'UIInterfaceOrientationLandscapeRight,UIInterfaceOrientationLandscapeLeft' in bundle 'net.XXXXXXX'.)
2021-02-16 19:05:08.107 altool[14798:356148] *** Error: code 1091 (App Store operation failed. Invalid Bundle. Your app supports Multitasking on iPad, so you must include the UILaunchStoryboardName key in your bundle, 'net.XXXXXXXX'. Learn more (https://developer.apple.com/library/content/documentation/General/Reference/InfoPlistKeyReference/Articles/iPhoneOSKeys.html#//apple_ref/doc/uid/TP40009252-SW40).)

reuk · February 16, 2021, 6:36pm

Did you wipe your build directory after enabling REQUIRES_FULL_SCREEN? There’s a chance the plist wasn’t being regenerated if you were reusing an existing build dir. Also, can you confirm that you have all orientations enabled for both IPAD_SCREEN_ORIENTATIONS and IPHONE_SCREEN_ORIENTATIONS?

If I check the plist in the app bundle after building the DemoRunner for iOS, I can see that the UISupportedInterfaceOrientations and UILaunchStoryboardName keys are set to the expected values.

bastienc · February 16, 2021, 9:04pm

My cmake file seems correct. I checked with master branch and there were no warnings regarding fullscreen not beeing set. I reproduce the issue with the develop branch. What branch were you using ?

juce_add_plugin(${PROJECT_NAME}
    PRODUCT_NAME  ${PROJECT_NAME}
    VERSION ${VERSION}
    BUNDLE_ID ${VENDOR_IDENTIFIER}.${PROJECT_NAME}
    COMPANY_NAME "XXX"
    COMPANY_WEBSITE "XXX"
    COMPANY_EMAIL "XXX"
    COMPANY_COPYRIGHT ${COPYRIGHT}
    IS_SYNTH FALSE
    NEEDS_MIDI_INPUT TRUE
    NEEDS_MIDI_OUTPUT FALSE
    IS_MIDI_EFFECT FALSE
    EDITOR_WANTS_KEYBOARD_FOCUS TRUE
    PLUGIN_MANUFACTURER_CODE "XXX "
    PLUGIN_CODE "XXXX"
    AAX_CATEGORY AAX_ePlugInCategory_Reverb
    COPY_PLUGIN_AFTER_BUILD TRUE
    SUPPRESS_AU_PLIST_RESOURCE_USAGE TRUE
    CUSTOM_XCASSETS_FOLDER ${CMAKE_CURRENT_SOURCE_DIR}/Images.xcassets
    TARGETED_DEVICE_FAMILY "1,2" # iPhone and iPad
    REQUIRES_FULL_SCREEN TRUE
    IPAD_SCREEN_ORIENTATIONS UIInterfaceOrientationLandscapeRight UIInterfaceOrientationLandscapeLeft
    IPHONE_SCREEN_ORIENTATIONS UIInterfaceOrientationLandscapeRight UIInterfaceOrientationLandscapeLeft
    BACKGROUND_AUDIO_ENABLED TRUE
    APP_GROUPS_ENABLED TRUE
    ICLOUD_PERMISSIONS_ENABLED TRUE
    MICROPHONE_PERMISSION_ENABLED TRUE
    MICROPHONE_PERMISSION_TEXT "${PROJECT_NAME} requires microphone usage"
    APP_GROUP_IDS group.${VENDOR_IDENTIFIER}.${PROJECT_NAME}
    PLIST_TO_MERGE ${PLIST_TO_MERGE}}
    FORMATS ${FORMATS})

bastienc · February 16, 2021, 9:59pm

This commit is linked with my trouble:

iOS: Added options to the Projucer and CMake to set UIRequiresFullScreen .plist option 65f2de3de ed <eddavies95@gmail.com> 5 Feb 2021 at 12:34

it seems that requiresFullScreen is never true no matter what I set in REQUIRES_FULL_SCREEN

 if (iOS)
    {
        if (type != ProjectType::Target::AudioUnitv3PlugIn)
        {
            if (statusBarHidden)
                addPlistDictionaryKey (*dict, "UIStatusBarHidden", true);

            if (requiresFullScreen) //works if commented (old behavior)
                addPlistDictionaryKey (*dict, "UIRequiresFullScreen", true);

            addIosScreenOrientations (*dict);
            addIosBackgroundModes (*dict);
        }

reuk · February 16, 2021, 10:16pm

You need all four user interface orientations enabled, in order for multitasking to work. It looks like you’ve only enabled the landscape orientations, so you should enable the portrait orientations too.

I’ve been using the develop branch. Multitasking definitely works in the DemoRunner on that branch.

bastienc · February 16, 2021, 10:56pm

I do not intend to use the multitasking, My app only supports landscape.
I am addressing a potential regression when the multitasking support has been provided within juce cmake.

The issue is when multitasking is disabled because UIRequiresFullScreen is never (beeing true or false) set in the plist.

reuk · February 17, 2021, 11:06am

Thanks, I understand the issue now. I’ll get that patched on develop.

reuk · February 18, 2021, 1:13pm

peteatjuce · February 23, 2021, 9:45am

Hi @reuk

I’m sorry to report I’ve (just) now come across a problem with this change.

The issue is that orientation changes work fine on iOS Simulators, but we’ve found they’ve stopped responding to orientation changes with physical devices. It looks like some sort of timing related issue in the Juce code. Maybe there is a recent iOS change that has aggravated this?

Anyhow, I’ve this morning had to revert back to my own split screen handling code, where orientation changes are all picked up fine (and split-screen still works, of course).
juce_ios_WIndowing.mm
juce_ios_UIViewComponentPeer.mm

If you’d like my copy of the code, just send me a mail, I’d be happy to give it to you of course!

Best wishes,

Pete

iOS Multitasking

Purchase

Discover

Learn

Support

About

Events