Optimize MIDI clip UI rendering

Tracktion Engine
1

Hi !

I’m looking to optimize the UI rendering code of MIDI clips in my app.

Rather than blindly drawing notes on paint() calls, I was thinking to pre-render them in an juce::Image and simply draw the image in paint(). The Image could re-render when the te::MidiClip state changes.

Another concern is having to many ValueTree::Listeners – I don’t think having one listener per MIDI clip component is ideal, so I’m happy to hear your suggestions on this too.

Thank you in advance,

Best,

E.

2

In one of my products, before the Juce framework, on the very first generation iPad, I also suffered with midi clips and midi editor, so that I could draw more than 6 thousand notes, because if, for example, you download and import a Beethoven sonata :), it may well exceed this amount.

  1. I just came to the conclusion that I would draw notes through OpenGL
  2. I don’t think drawing each clip individually is a good idea. It should be a single draw call, because user can have thousands of small clips (if your app allows it) I would draw the entire track. Or if it’s OpenGL, then whole midi view.

I was thinking to pre-render them in an juce::Image and simply draw the image in paint() .

Why not to use simply Component::setBufferedToImage? And then if you need to change a state, call repaint();

this->setBufferedToImage(true); // set in the constructor

In any case when you need to change state, you will draw your notes to the Image, so you don’t win anything here and setBufferedToImage make it easy for you (in this case the draw call will not be called if you redraw other components)

2 Likes
3

Before you spend a long time rewriting things, make sure you profile a release build to find out where the time is actually being spent. For things that move around a lot (like clips on a timeline or a piano roll), buffering to an image could well be slower as you’ll constantly have to redraw the image and then blit that to the graphics context rather than the draw calls directly (which are async on some OSes like macOS).

FYI, we simply draw the MidiLists in Waveform.

1 Like
4

Good call. I was looking to use Perfetto for profiling (not a big fan of Instruments on macOS), through this article Using Perfetto with JUCE for dsp and UI performance tuning · Melatonin

So … in Waveform you simply draw the notes everytime on paint() calls ?

5

Yes. Personally I find Instruments on macOS excellent.
Even better if you combine it with some signposts.

But more generally it’s going to tell you where your CPU time is spent. If you focus all your effort on improving the drawing but it turns out your CPU is spending ages doing something else, you won’t have made any improvements.

2 Likes
6

Yeah, I guess Instruments would be OK to profile the drawing code, but I’m geniunly curious about Perfetto. I really miss Shark & VTune !

7

You can still use VTune can’t you? I’ve used that on Linux.

1 Like
8

Yeah, I just remember about it.

Anyway, I can comfirm setBufferedToImage(true) is far less optimal compared to simply drawing the events in paint().

9

Anyway, I can comfirm setBufferedToImage(true) is far less optimal compared to simply drawing the events in paint().

setBufferedToImage will not be optimal if you are redrawing a whole piece of a large view.
For example, in a synthesizer, I use it by zones:
-osc,
-Envelope,
-keys
That is, when I change the values in Osc, then repaint is not called in Envelope and Keys.

oscView.setBufferedToImage(true);
envelopeView.setBufferedToImage(true);
keysView.setBufferedToImage(true);

then I can call for examle:

osc.getGainControl().setValue(newValue);

the buffer image will be updated only for osc (i.e. all child components in the osc trigger repaint method), while for the rest it will be drawn what is still in the “envelope” and “keys” buffers.

That is, if you try to set it for each midi clip or for the whole screen view - then yes it will not be optimal. Since in this case you are essentially drawing in Image and then this image is drawing to the component. Also it uses more memory.