VST crash when modal dialog open

I have written a VST plugin that can host other VST plugins.
When my AudioPluginsManager (a modal dialog) is open, and the user then destroys the plugin, the first thing that happens is:

jassert (! recursionCheck); in the vst wrapper close() function.

After that I get exceptions. I am destroying the Plugin’s editor in the plugin’s destructor.

Yeah, it’s a known problem that’s a bit tricky to fix. The best solution is just not to make your dialog box modal…

I’m using a PluginListComponent, so there are automatically modal windows involved. Is there no workaround?

Not really - the juce modal windows aren’t seen as modal by the host, so you can’t stop them messing about with things while it’s open. The only way would be to go into a proper win32 modal loop while it’s showing, but that plays havoc with things like menus, and on the mac it causes an even bigger mess because of carbon/cocoa event loops. I never quite figured out a foolproof way of getting modalness to work the way it should.

I don’t know exactly how JUCE all works internally, but why can’t you just (in the vstrwrapper) close all the open modal JUCE windows there in the deleteEditor() function that gets called on effEditClose? Like close them and wait 'til they’re destroyed then continue.

Maybe this is a stupid question, but I don’t exactly see the problem of doing this, because JUCE knows what modal windows are open, so it can close them again?

yeah, that’d nice, but impossible! Think about the state of the stack at the point where deleteEditor is called - it’s sitting somewhere inside your own code, inside a call to runModal(), and the only way out involves coming back down the stack. I tried to add a workaround that cancels the modal windows and defers the deletion until later with an event, but the host might not hang around long enough to let that complete before zapping the plugin.

OK, and if it’s a non-modal window, what’s the state of the stack at the point when deleteEditor() is called?

[ I wouldn’t be bugging you if I had some other choice. The thing is I just rewrote the whole PluginListComponent to be non-modal, but at the some point, where the user can specify a VST directory, there is again a native FileChooser involved which is modal, and so it’s f***ed ].

Added:

I don’t if you know ( 8) ), but if you want your windows such as native Windows File Browsers to be modal with respect to the host, you have to pass them the host’s HWND as owner HWND. If you do pass the host’s HWND, the user will not be able to click into the host as long as he didn’t select a file/directory.

If everything’s non-modal, then it should be fine.

Not sure what to do about the file chooser…

Well, if you would add another parameter to the FileChooser, called “parentComponent”, then the FileChooser could get the top level HWND and that’s it. It would be good anyway for all kind of OS-modal dialogs to be able to specify the parent.

That’s at least how I did it. It works. I can’t click “into the host” anymore. It would require modifying JUCE a little bit, but hey, it would be worth it. I lost hours today modifying JUCE for my own purposes.