The documentation of ChildProcessCoordinator / ChildProcessWorker makes me wonder if it is possible to create multiple instances of ChildProcessCoordinator that each command a ChildProcessWorker at the same time (e.g. launched by a ThreadPool).
Documentation says processUID must match at both ends. However, if all coordinators had the same processUID, how would a worker know to which it belongs?
I can think of using processUID [1…6] corresponding to the ThreadPool’s size. The worker app would then simply attempt initialiseFromCommandLine() with each index until it succeeds. Is that aviable approach?
First I restructured the procedure a bit in order to avoid the convoluted way KnownPluginList is currently involved in the scanning process. To separate responsibilities and eliminate the confusing back and forth, I created a PluginFinder class that searches for all plugins (identifiers) per format in advance and decides which need a (re)scan based on the current KnownPluginList.
This list is fed as jobs into a ThreadPool with 8 threads (one job per plugin). So this could be thousands of jobs depending on how many plugins were found. When a job is due it creates a ChildProcessCoordinator that launches a ChildProcessWorker (separate executable) which scans that plugin and reports back XML (copied the code from AudioPluginHost).
This seems to work and is much faster than a serial scan loop. MUCH faster.
I still wonder if it’s a good idea to have all workers share the same pipe, though. Something smells fishy with that. Indeed not every runs equally smooth, so there are remaining issues that may be related to this.
IPC is very difficult to debug, so I will just try and add an additional parameter instanceId (an integer incremented indefinitely) to ChildProcessCoordinator and see if that works better. Will report back.
Has anyone done this before? I mean, scanning in parallel is pretty much standard nowadays. I have seen quite a few users with 2,000+ plugins and don’t think this is so unusual.
I should have examined the code more thoroughly. ChildProcessCoordinator already uses a random pipe name in addition to the prefix and passes that to the worker via command line, so everything is fine.
Sorry for the noise. This happens when you’re stressed and impatient to get something done asap.
I still hope the previous post is useful as an inspiration how to implement this.
