RSA speed is mostly about the time it takes to raise a number to a power. This is roughly proportional to the log of the power…
so for example, to compute x^3, you have to do two multiplies. To compute x^256, you need to do 8 multiplies.
RSA encoding can (mostly) pick one of the powers, almost always for the public key, and almost always for efficiency. So x^3 is a popular choice. So is 65537, which is also efficient to compute.
However, the PRIVATE exponent is always long, and for a 2048 bit key, will tend to have an exponent with close to 2048 bits.
It’s roughly 2000 times slower to raise a number to a power with 2000 digits than it is to raise it to a power with 1 digit.
So, your public speed of 37ms is computing x^3 (notice the 3 in the private key). The private decode, with 1000 bit exponent really is about 1000 times more work, so your 37ms and 13 sec timings are in the right ballpark.
Annoying? Yep, but that’s part of the cost of RSA. The public part is fast only because it’s CHOSEN to be specifically efficient. It’s not a JUCE problem in general.
The slowness of RSA is one reason people often use RSA to encrypt a large random key to a fast symmetric cypher, and then encrypt the actual message with the symmetric cypher.