How to access score data in Sibelius in a way similar to NotePerformer?

General JUCE discussion
1

I’m working on an AU plugin using JUCE that aims to analyze musical data from Sibelius (and eventually Finale, Dorica, MusScore, etc.) during playback.

Ideally, I want the plugin to access information like notes, dynamics (p, pp, crescendos), and performance text (pizzicato etc.) from the currently playing staff/staves so I can send it to a group of functions for processing/analysis. As far as I can figure, this is how NotePerformer functions.

Issue: While JUCE provides a great framework for AU plugins, I haven’t found documented ways to directly access Sibelius score data within the plugin itself.

My current hypothetical approach involves:

  • Developing a separate Sibelius script (VBScript or Javascript) that triggers on playback start.

  • The script extracts relevant data and converts it to JSON format.

  • The AU plugin would then access the data (through shared memory or a file) and process it for further analysis.

Questions:

  • Are there any alternative approaches for an AU plugin to access Sibelius score data during playback? (Even if they involve undocumented methods, I’d appreciate insights)

  • Has anyone explored similar functionalities using JUCE and Sibelius?

  • Are there any limitations or potential issues with the approach I outlined above?

Any advice or suggestions would be greatly appreciated!

2

From the NotePerformer website:

Screenshot 2024-05-23 at 14.05.39
Screenshot 2024-05-23 at 14.05.39789×360 41.2 KB

They simply add a lookahead of one second. They don’t analyse the whole score.

I was comtemplating similar ideas like you did and came to the conclusion, it would b best for the user to export the score as musicXML and import it into your playback engine.

3

I knew that much (but thanks). I’m trying to find out exactly, technically, logistically HOW do you get the data from the score into JUCE. I figure I’ll only need to load a couple measures at a time as playback rolls.

4

That is one solution, the other one is ARA, but I don’t think Sibelius supports ARA.

When you do it as plugin, you only get the midi. You don’t get to see the articulations from the score, which I as orchestra player find way more meaningful than the midi.

5

It must be available somehow… NotePerformer is DOING it… I wonder, could they be converting all the text and symbols to MIDI first?

At this stage I’d love to be able to just get MIDI notes into JUCE from Sibelius.

6

I thought from their post they are simply recording the midi stream of the playback.
But I am eager to learn, if somebody else has an idea.

7

I know VST3 doesn’t give you anything like notation or MIDI (VST3 doesn’t support MIDI at all), so that’s probably not what they’re using on Windows. What’s interesting though is that Finale, Dorico, and Sibelius all have custom scripting and those are the only things explicitly supported by NotePerformer.

So you might want to try seeing what MIDI your plugin gets from Sibelius to see what is coming through, and looking through their docs on their scripting language to see what’s available. The 1 second buffer is a very long time, possibly to run their script through the entire score before playing back with the plugin.