I’ve investigated this a bit now. I created a plugin project using JUCE 5.2.1 and the corresponding Projucer version. I created a Reaper project using this version of the plugin. Then, I wrote a CMakeLists for the same project using JUCE 6, built and installed the plugin, and then tried to re-open the Reaper project.
At first I was seeing the same issue that you’re seeing. After some debugging I realised that the VST3 UIDs were different for the different plugin versions.
When I modified the CMakeLists to include JUCE_VST3_CAN_REPLACE_VST2=1 in target_compile_definitions, the UIDs matched again, and Reaper loaded the JUCE 6 plugin correctly. I also needed to call juce_set_vst2_sdk_path pointing to a local copy of the VST2 SDK in order to build with this flag enabled. Can you check whether you’re using the JUCE_VST3_CAN_REPLACE_VST2 flag?