Is it safe to swap the MemoryBlock of an OSC blob received?

Given is the following situation:

There is only one OSCReceiver::ListenerWithOSCAddress for a particular OSC Address. Now in the oscMessageReceived callback I do something like that to prevent memory copies:

void oscMessageReceived (const juce::OSCMessage& message) override
{
    auto blockReceived = const_cast<juce::MemoryBlock&> (message[0].getBlob());

    // myMemoryBlock is a class member of my OSCReceiver::ListenerWithOSCAddress implementation
    myMemoryBlock.swapWith (blockReceived);

    someFunctionThatWillDoSomethingWithMyMemoryBlock();
}

Looking at the codebase I can’t find any problem with this approach as long as this is the only listener and the blob is only accessed here once. The OSCMessage containing the OSCArgument that holds this memory block should go out of scope after the listener callback anyway. However, did I overlook anything here that would prevent me from safely doing this under the circumstances described above?