I make plugins for a host who only expects me to show a true modal dialog (Premiere Pro). So its “Show settings” callback is called with the host already fully disabled; I get a native handle to the host window to parent myself to, I’m expected to show a modal dialog to do my thing, blocking execution on the dialog, and only to return once the dialog is dismissed. So none of that semi-modal async embedded panel VST plugin stuff 
That said I had the hardest time getting true modal dialogs to work properly with Juce. And not only do I need to show modal dialogs on the host, but I also need modals on my modals, since that one main modal dialog is a switchboard to all sort of functionality of my plugin. I just want to build a multi-tier dialog-based app in my plugin.
The problems with using the default Juce modal functionality:
- I can specify the parent dialog on OSX, but it then makes the modal dialog a subview of the host’s view, effectively embedding the dialog as a subcontrol in the host window.
- Not specifying a parent on OSX makes the dialog disconnected from the host, and the user can just click on and/or Ctrl-Tab to the disabled host and my dialog will go behind it (forgot which one it was). This can be solved by making my dialog a topmost window, but then it’s topmost for all other apps in the system, which is also undesirable.
- Specifying another Juce dialog as the parent of another Juce dialog also has this sub-can-go-behind-parent nuisance, on both Windows and OSX.
I now however have some code ready that seems to play nicely on both Windows and OSX, with one remaining minor nuicance on Windows. What I came up with is a C_ModalDialog class, together with a helper C_ParentWindow class which allows the C_ModalDialog to be parented to both Juce components/dialogs and native window handles. It wraps a Juce::DialogWindow, and I’m using a modified version of the innards of Juce’s LaunchOptions to get it going. As a solution for the OSX topmost hack I just parent the new Juce window to the host window, and as a solution to Juce subdialogs being able to go behind their Juce parent dialogs I just cut the link between; I get the Juce parent dialog’s native window handle and treat it just like I do the host window handle.
The C_ParentWindow is a simple wrapper that enables me to transform both native handles and Juce dialog pointers into the same “native parent window handle + possible parent Juce component” data, so that I can parent my dialogs to both Juce dialogs and native handles the exact same way.
class C_ParentWindow {
// the native handle to use for this parent window
void* m_handle = nullptr;
// the associated Juce component, if any
juce::Component* m_componentPtr = nullptr;
// sets the parent window to be either the given native handle or the given Juce component
C_ParentWindow(void* parentWindowHandle) {
m_componentPtr = nullptr;
m_handle = parentWindowHandle;
}
C_ParentWindow(juce::Component* parentComponentPtr) {
m_componentPtr = parentComponentPtr;
#ifdef __APPLE__
m_handle = [(NSView*)parentComponentPtr->getWindowHandle() window];
#else
m_handle = parentComponentPtr->getWindowHandle();
#endif
}
};
class C_ModalDialog : public juce::DialogWindow {
int LaunchModal(juce::Component* dialogContentPtr, C_ParentWindow& parentWindow);
// set our options
setUsingNativeTitleBar(true);
setAlwaysOnTop(false);
setContentNonOwned(dialogContentPtr, true);
centreAroundComponent(parentWindow.m_componentPtr, getWidth(), getHeight());
setResizable(false, false);
// bring us to life, ensuring we're parented under the parent
#ifdef __APPLE__
addToDesktop(getDesktopWindowStyleFlags(), nullptr);
NSWindow* parentWindowHandle = (NSWindow*)parentWindow.m_handle;
NSWindow* usAsWindowHandle = [(NSView*)getWindowHandle() window];
[parentWindowHandle addChildWindow:usAsWindowHandle ordered:NSWindowAbove];
#else
addToDesktop(getDesktopWindowStyleFlags(), parentWindow.m_handle);
#endif
// transit to a modal state
enterModalState(true, nullptr, true); // last 'true' means Juce will delete us on exit of dialog
// show the dialog
int result = runModalLoop();
// decouple the dialog from the host
#ifdef __APPLE__
[parentWindowHandle removeChildWindow:usAsWindowHandle];
#endif
// ensure our parent has focus now
// workaround for Windows -- if not done so, Juce effectively performs an alt-tab if we return to the host
// window when we stacked 2 or more subdialogs on each other
#ifdef _WIN32
HWND parentWindowHandle = (HWND)parentWindow.m_handle;
SetForegroundWindow(parentWindowHandle);
#endif
// and return the result
return result;
}
// gets the window style flags to use
int getDesktopWindowStyleFlags() const override {
// for windows: ensure the 'add to taskbar' flag is not set so we don't end up there and in the alt-tab list
// FIXTHIS: has other undesired side effect of the content not fitting in the window anymore &&
// the window not having our icon anymore
return juce::DialogWindow::getDesktopWindowStyleFlags();// & ~juce::ComponentPeer::windowAppearsOnTaskbar;
}
};
The above code is a stripped down version of what I have (it is a base class for all my modal dialogs, and handles all sorts of commonalities for my modal dialogs), but the code should work as adverised I think.
This seems to work nicely on OSX, and on Windows the only beef I still have with it is that it shows a taskbar icon for every modal opened, but I can live with that if need be. However, is the above code good enough? I’m not a true OSX developer, so have I dropped the ball on something there? Any caveats I overlooked or OS versions I’m now incompatible with? If anyone could add their thoughts about it, that would be great!