Shared Variables Among different plugins

Audio Plugins
1

Hi, I’m waiting because I have a really big problem.

There is a way to have a static variable shared among different plugins instances of different types?

so for example

struct Shared
{
static int variable;
};

class PluginA
{
Shared shared;
};

class PluginB
{
Shared shared;
};

obviously in projucer I created two project first is PluginA and second is PluginB and both once launched must see only a shared instance of shared.variable.

up to now if I launch in cubase for example 2 instance of PluginA shared.variable is shared among the two, but not among for example an instance of PluginA and one of PluginB

2

static values are shared only within the same dll.
So If your two plugins are built separately, they will not share static values.

While most plugin API support having more than one plugin in the same dll, I don’t think JUCE does.

An alternative for communicating between DLLs might be to use a memory mapped file.

4 Likes
3

Indeed, @JeffMcClintock’s advice is good .. here’s something I have in my lab folder that might be useful:

(EDIT: I decided I need to use this myself personally, so I fleshed it out quite a bit and made it a bit more robust .. updated code below - to test, run two copies of the ‘smm’ binary ..)

#include <string>
#include <stdexcept>
#include <cstdint>
#include <iostream>
#include <vector>
#include <cstring>

#ifdef _WIN32
#include <windows.h>
#else
#include <sys/mman.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>
#include <errno.h>
#include <signal.h>
#endif

class SharedMapMemory {
public:
    SharedMapMemory(const std::string& name, size_t size, bool create = true)
        : m_name(name), m_size(size), m_ptr(NULL), m_handle(
#ifdef _WIN32
            INVALID_HANDLE_VALUE
#else
            -1
#endif
        ), m_is_creator(create) {
        if (size == 0) {
            throw std::invalid_argument("Shared memory size must be greater than 0");
        }
        std::cerr << "Attempting to " << (create ? "create" : "open") 
                  << " shared memory '" << name << "' with size " << size << std::endl;
        open(create);
    }

    ~SharedMapMemory() {
        close();
    }

#if __cplusplus >= 201103L
    SharedMapMemory(const SharedMapMemory&) = delete;
    SharedMapMemory& operator=(const SharedMapMemory&) = delete;
    SharedMapMemory(SharedMapMemory&& other) noexcept
        : m_name(std::move(other.m_name)), m_size(other.m_size),
          m_ptr(other.m_ptr), m_handle(other.m_handle), m_is_creator(other.m_is_creator) {
        other.m_ptr = NULL;
        other.m_handle = 
#ifdef _WIN32
            INVALID_HANDLE_VALUE;
#else
            -1;
#endif
        other.m_is_creator = false;
    }
    SharedMapMemory& operator=(SharedMapMemory&& other) noexcept {
        if (this != &other) {
            close();
            m_name = std::move(other.m_name);
            m_size = other.m_size;
            m_ptr = other.m_ptr;
            m_handle = other.m_handle;
            m_is_creator = other.m_is_creator;
            other.m_ptr = NULL;
            other.m_handle = 
#ifdef _WIN32
                INVALID_HANDLE_VALUE;
#else
                -1;
#endif
            other.m_is_creator = false;
        }
        return *this;
    }
#endif

    void* get() const { return m_ptr; }
    size_t size() const { return m_size; }

private:
    std::string m_name;
    size_t m_size;
    void* m_ptr;
#ifdef _WIN32
    HANDLE m_handle;
#else
    int m_handle;
#endif
    bool m_is_creator;

    void open(bool create) {
#ifdef _WIN32
        DWORD creationDisposition = create ? CREATE_ALWAYS : OPEN_EXISTING;
        m_handle = CreateFileMappingA(
            INVALID_HANDLE_VALUE,
            NULL,
            PAGE_READWRITE,
            0,
            static_cast<DWORD>(m_size),
            m_name.c_str()
        );
        
        if (m_handle == NULL) {
            throw std::runtime_error("Failed to create/open file mapping: " + 
                                   std::to_string(GetLastError()));
        }

        m_ptr = MapViewOfFile(
            m_handle,
            FILE_MAP_ALL_ACCESS,
            0,
            0,
            m_size
        );

        if (m_ptr == NULL) {
            CloseHandle(m_handle);
            m_handle = INVALID_HANDLE_VALUE;
            throw std::runtime_error("Failed to map view of file: " + 
                                   std::to_string(GetLastError()));
        }
#else
        int flags = O_RDWR;
        if (create) {
            flags |= O_CREAT | O_EXCL;
            m_handle = shm_open(m_name.c_str(), flags, S_IRUSR | S_IWUSR);
            if (m_handle == -1) {
                if (errno == EEXIST) {
                    throw std::runtime_error("Shared memory already exists");
                }
                throw std::runtime_error("Failed to create shared memory: " + 
                                       std::string(strerror(errno)));
            }

            std::cerr << "Creating with fd " << m_handle << " and size " << m_size << std::endl;
            if (ftruncate(m_handle, m_size) == -1) {
                int saved_errno = errno;
                ::close(m_handle);
                shm_unlink(m_name.c_str());
                m_handle = -1;
                throw std::runtime_error("Failed to set shared memory size: " + 
                                       std::string(strerror(saved_errno)));
            }
            struct stat sb;
            if (fstat(m_handle, &sb) == -1) {
                throw std::runtime_error("Failed to verify size: " + 
                                       std::string(strerror(errno)));
            }
            std::cerr << "Created shared memory with actual size: " << sb.st_size << std::endl;
            m_size = sb.st_size;
        } else {
            m_handle = shm_open(m_name.c_str(), flags, S_IRUSR | S_IWUSR);
            if (m_handle == -1) {
                throw std::runtime_error("Failed to open shared memory: " + 
                                       std::string(strerror(errno)));
            }
            struct stat sb;
            if (fstat(m_handle, &sb) == -1) {
                int saved_errno = errno;
                ::close(m_handle);
                m_handle = -1;
                throw std::runtime_error("Failed to get shared memory size: " + 
                                       std::string(strerror(saved_errno)));
            }
            m_size = sb.st_size;
            std::cerr << "Opened existing shared memory with size: " << m_size << std::endl;
        }

        m_ptr = mmap(NULL, m_size, 
                    PROT_READ | PROT_WRITE, 
                    MAP_SHARED, 
                    m_handle, 
                    0);

        if (m_ptr == MAP_FAILED) {
            int saved_errno = errno;
            ::close(m_handle);
            if (m_is_creator) shm_unlink(m_name.c_str());
            m_handle = -1;
            throw std::runtime_error("Failed to map shared memory: " + 
                                   std::string(strerror(saved_errno)));
        }
#endif
    }

    void close() {
        if (m_ptr) {
#ifdef _WIN32
            UnmapViewOfFile(m_ptr);
            if (m_handle != INVALID_HANDLE_VALUE) {
                CloseHandle(m_handle);
            }
#else
            if (munmap(m_ptr, m_size) == -1) {
                std::cerr << "Warning: Failed to unmap memory: " << strerror(errno) << std::endl;
            }
            if (m_handle != -1) {
                if (::close(m_handle) == -1) {
                    std::cerr << "Warning: Failed to close handle: " << strerror(errno) << std::endl;
                }
                if (m_is_creator) {
                    if (shm_unlink(m_name.c_str()) == -1) {
                        std::cerr << "Warning: Failed to unlink " << m_name << ": " 
                                << strerror(errno) << std::endl;
                    }
                }
            }
#endif
            m_ptr = NULL;
            m_handle = 
#ifdef _WIN32
                INVALID_HANDLE_VALUE;
#else
                -1;
#endif
        }
    }
};

struct SharedData {
    int pid_count;
    int pids[100];
};

int main() {
    try {
        const size_t REQUESTED_SIZE = sizeof(SharedData);
        std::cerr << "SharedData size: " << REQUESTED_SIZE << std::endl;
        if (REQUESTED_SIZE <= 0) {
            throw std::runtime_error("Shared memory size is invalid");
        }
        
        bool is_creator = true;
        SharedMapMemory* shm = NULL;
        try {
            shm = new SharedMapMemory("/MySharedMem", REQUESTED_SIZE, true);
        }
        catch (const std::exception& e) {
            std::cerr << "Creator failed: " << e.what() << std::endl;
            is_creator = false;
            try {
                shm = new SharedMapMemory("/MySharedMem", REQUESTED_SIZE, false);
            }
            catch (const std::exception& e2) {
                std::cerr << "Joiner also failed: " << e2.what() << std::endl;
                throw;
            }
        }

        SharedData* data = static_cast<SharedData*>(shm->get());
        
        if (is_creator) {
            data->pid_count = 0;
            memset(data->pids, 0, sizeof(data->pids));
        }

#ifdef _WIN32
        int my_pid = GetCurrentProcessId();
#else
        int my_pid = getpid();
#endif

        bool pid_added = false;
        for (int i = 0; i < 100; i++) {
            if (data->pids[i] == 0 && !pid_added) {
                data->pids[i] = my_pid;
                data->pid_count++;
                pid_added = true;
            }
        }

        while (true) {
            std::cout << "Process " << my_pid << " sees PIDs: ";
            std::vector<int> active_pids;
            for (int i = 0; i < 100; i++) {
                if (data->pids[i] != 0) {
                    active_pids.push_back(data->pids[i]);
                    std::cout << data->pids[i] << " ";
                }
            }
            std::cout << "(Total: " << data->pid_count << ")" << std::endl;

            for (size_t i = 0; i < active_pids.size(); i++) {
#ifdef _WIN32
                HANDLE h = OpenProcess(PROCESS_QUERY_LIMITED_INFORMATION, FALSE, active_pids[i]);
                if (h == NULL) {
                    for (int j = 0; j < 100; j++) {
                        if (data->pids[j] == active_pids[i]) {
                            data->pids[j] = 0;
                            data->pid_count--;
                            break;
                        }
                    }
                }
                else {
                    CloseHandle(h);
                }
#else
                if (kill(active_pids[i], 0) == -1 && errno == ESRCH) {
                    for (int j = 0; j < 100; j++) {
                        if (data->pids[j] == active_pids[i]) {
                            data->pids[j] = 0;
                            data->pid_count--;
                            break;
                        }
                    }
                }
#endif
            }

#ifdef _WIN32
            Sleep(1000);
#else
            sleep(1);
#endif
        }

        delete shm;
    }
    catch (const std::exception& e) {
        std::cerr << "Error: " << e.what() << std::endl;
        return 1;
    }
    return 0;
}

To compile and test (assuming you saved the code as ‘smm.cpp’):

g++ -std=c++11 smm.cpp -o smm

Run results:

# Terminal 1
# ./smm
SharedData size: 404
Attempting to create shared memory '/MySharedMem' with size 404
Creator failed: Shared memory already exists
Attempting to open shared memory '/MySharedMem' with size 404
Opened existing shared memory with size: 16384
Process 28287 sees PIDs: 28142 28287 (Total: 2)
Process 28287 sees PIDs: 28287 (Total: 1)
Process 28287 sees PIDs: 28287 (Total: 1)
Process 28287 sees PIDs: 28287 (Total: 1)
Process 28287 sees PIDs: 28291 28287 (Total: 2)
Process 28287 sees PIDs: 28291 28287 (Total: 2)
Process 28287 sees PIDs: 28291 28287 (Total: 2)
Process 28287 sees PIDs: 28291 28287 (Total: 2)
Process 28287 sees PIDs: 28291 28287 (Total: 2)
Process 28287 sees PIDs: 28287 (Total: 1)
Process 28287 sees PIDs: 28287 (Total: 1)
Process 28287 sees PIDs: 28287 (Total: 1)

# Terminal 2:
./smm
SharedData size: 404
Attempting to create shared memory '/MySharedMem' with size 404
Creator failed: Shared memory already exists
Attempting to open shared memory '/MySharedMem' with size 404
Opened existing shared memory with size: 16384
Process 28291 sees PIDs: 28291 28287 (Total: 2)
Process 28291 sees PIDs: 28291 28287 (Total: 2)
Process 28291 sees PIDs: 28291 28287 (Total: 2)
Process 28291 sees PIDs: 28291 28287 (Total: 2)
^C

(Disclaimer: untested on Windows/iOS …)

EDIT2: I decided to use this in a project, so I put it in a repo:

5 Likes
4

or you could put your shared variables in a DLL, and load the same DLL from both plug-in types? Disclaimer, I’ve never tried this in practice, but I am aware that both JUCE and CHOC offer mechanisms to load dynamic libraries

Or you could use some sort of inter-process communication, if your model is more the kind of “one server” (e.g. a license manager that runs as a standalone app) and multiple clients (the plug-ins)

1 Like
5

Within a plugin, you can also use JUCE’ built-in SharedResourcePointer:

    SharedResourcePointer<SharedParams> sharedParams;

.. to share a struct between multiple instances of your plugin.

It should be noted also that Shared memory-mapped structs are not necessarily a ‘safe’ way to do things in this era of sandboxes and isolation .. better is to use the Inter-process mechanisms that are available for your chosen target platforms .. and of course, if JUCE’ own MemoryMappedFile works for your use case, its preferable, also…

6

IPC methods are your only option. Modern DAWs can load your plugin instances in seperate processes which makes the static/shared DLL methods not work anymore.

2 Likes
7

I believe most DAWs will put all the same plugin in a single process space – so SharedResourcePointer should work within the same plugin.. but not across plugins which will likely be in a separate process.

Rail

1 Like
8

Hi everyone,

Has anybody experimented with this and can offer some insights?

I am also looking for ways to share a variable between different plugins across Windows or Mac DAWs. Do you reckon it is possible via named pipes between those plugins? Or is it going to need a local service to synchronize those queries?

Alternatively do you suggest other IPC methods?

9

Memory mapped files with proper synchronization is the best approach.

JUCE’ MemoryMappedFile and InterprocessLock are within the grasp of the average JUCE project.

In a more sophisticated setting, having your own ‘helper agent’ which accepts socket connections from your other plugins might be a fruitful approach as well, albeit with heavier load for the user.

[1] JUCE: MemoryMappedFile Class Reference
[2] JUCE: InterProcessLock Class Reference

#pragma once
#include <juce_core/juce_core.h>

// Manages shared memory for a single float variable
class SharedMemoryManager {
public:
    SharedMemoryManager(const juce::String& memoryName, const juce::String& lockName)
        : memoryName(memoryName), lockName(lockName) {
        // Initialize shared memory (size for one float)
        memoryFile = std::make_unique<juce::MemoryMappedFile>(
            memoryName, sizeof(float), juce::MemoryMappedFile::readWrite);

        // Initialize interprocess lock
        lock = std::make_unique<juce::InterprocessLock>(lockName);

        // Initialize the shared variable if it doesn't exist
        if (memoryFile->getData()) {
            juce::ScopedLock sl(*lock);
            *static_cast<float*>(memoryFile->getData()) = 0.0f; // Default value
        }
    }

    ~SharedMemoryManager() = default;

    bool write(float value) {
        if (!memoryFile->getData())
            return false;

        juce::ScopedLock sl(*lock);
        *static_cast<float*>(memoryFile->getData()) = value;
        return true;
    }

    bool read(float& value) {
        if (!memoryFile->getData())
            return false;

        juce::ScopedLock sl(*lock);
        value = *static_cast<float*>(memoryFile->getData());
        return true;
    }

private:
    juce::String memoryName;
    juce::String lockName;
    std::unique_ptr<juce::MemoryMappedFile> memoryFile;
    std::unique_ptr<juce::InterprocessLock> lock;
};
3 Likes
10

I have been having success linking parameters in Pro Tools across 2 different plugins using NamedPipes. The latency is not bad too.

1 Like
11

Has anybody experimented with this and can offer some insights?

Not in the context of juce or audio programming, but I successfully put an atomic variable in IPC shared memory on windows, and it worked great (this was in the context of sharing gui slider values between two instances of the same process)
Here’s the tutorial I used
Also if you do choose to go the shared memory route, you should note that it’s technically not okay to put an std::atomic in shared memory, even if it’s lock-free (because yada yada standard says nothing about different processes). It will almost certainly be fine in practice, but if you want to be 100% safe you should use platform specific atomic functions

Edit: ibisum’s solution is probably a lot better than mine

12

Hi ibisum, thanks a lot for sharing this.
I only have one concern. Is this going to work with sandboxed plugins? (e.g. in Bitwig). Does it work in Windows and Mac? In what context have you tried it?

13

I have used shared memory/IPC techniques successfully for decades in projects spanning a wide variety of contexts, but in the context of JUCE plugins I have only needed to use JUCE’ SharedResourcePointer for local sharing between multiple instances of the same plugin. You can see this in effect in the Austrian Audio PolarDesigner plugin, which provides the ability to copy parameter settings between up to 5 different instances of the plugin (the “Sync Group” feature).

Modern sandbox’ing techniques intend to limit IPC through gatekeeping methods, since data exfiltration through IPC mechanisms is a very common security nightmare, but as long as you use file-based IPC through memory mapped files to a common sandbox directory (e.g. ~/Music), with synchronisation primitives, you should be okay. Choose a sandbox-compliant shared directory for your mapped file and set up a timer to poll for updates to make it robust, and things should work fine.

Report in when you get it working and let us know how it goes. :wink:

1 Like
14

ibisum Thanks again! Very interesting. I need to share data between different plugins so I might find other issues. I’ll take it from here anyway!

1 Like
Interprocess communication (JUCE vs ZeroMQ)
15

Hey @Qfactor I was just working on this and thought I’d make you aware of some tweaks I made to JUCE to help with NamedPipes.

I’ve been doing this for a Mac-specific project, so I don’t know what surprises are waiting in other OSs.

2 Likes