Tips for fast thumbnail rendering


#1

Hi all, I was wondering if anyone has any tips for rendering waveforms in a DAW environment. I know there are a lot of DAW (or similar) developers out there and was hoping someone would be able to share some tips for drawing a large number of waveforms quickly, at different zoom levels and with constantly changing bounds i.e. scrolling with the transport.

At the moment I am using the AudioThumbnail class and caching these at a resolution of 64 samples per thumb sample. Then, my waveform is drawn as part of a component in a viewport timeline. To speed up the drawing I am clipping the bounds of the drawing time to Component::getVisibleArea() so obviously only the visible section of the waveform is drawn.

This works fairly well when the user is just scrolling around or zooming in and out but slows down when there are lots of visible thumbnails on the screen or when the window is constantly moving. Obviously this has to do with the fact that the all the thumbnail lines are drawn each time the window moves by as little as a pixel.

Now, what I’ve done in the past is to cache whole downscaled waveforms to images and then rescale sections of them when necessary but that doesn’t really fit well with this sort of application. I also need to be able to re-draw the waveforms at any zoom level and at the moment this crawls to a standstill when having to re-read from the actual file.

Here some ideas I have had:

  • Keep all the drawing off the main thread. I imagine this would work by when the zoom changes, a TimeSliceThread is used to draw the waveform into an image, notifying the main thread once it has drawn the required period so it can repaint itself.
    This will certainly make sure that the UI remains responsive but could lead to periods of blank waveforms while the rendering thread catches up with the viewed position. This wouldn’t work well for example if only the end section of a waveform is being viewed at a very high, constantly changing zoom level as the rendering thread would have to keep re-rendering the image from the beginning.
    This could also lead to very high memory consumption as extremely large images would be required at high zoom levels e.g. 3 pixels per sample would lead to 396900 wide image for a standard 3 min, 44100KHz file.
  • This could be improved by a selective render pattern, i.e. render the viewed part of the waveform first and then fill in the gaps around as the user is likely to scroll close by rather than jump around. However, this starts to get very complicated and in my experience you get inconsistencies when drawing using the AudioThumbnail class in sections as the low-res boundaries changes.
  • A compromise would be to render only the visible section on a background thread and repaint once that has been drawn. That should reduce the image size and amount of time to render the section but I am still worried that it won’t be fast enough to keep up with the transport.

I think this details some of the problems and I know there is a solution out there as Reaper for example does a fantastic job of quickly rendering, very zoomed in waveforms whilst following the transport cursor.

Any tricks, tips or general mockery of my current thinking are welcome.

Thanks again,
Dave.


#2

The following link may be of some help. It’s Mac focused but the Juce Graphics system pretty much resembles Quartz/CoreGraphics anyway so that shouldn’t be an issue.

http://supermegaultragroovy.com/2009/10/06/drawing-waveforms/

TheVinn has turned into a graphics wizard, maybe he has tackled something similar?

Hope any breakthroughs you make find their way into your awesome dRowAudio Juce module.


#3

Hi sonic, thanks for the reply, I’ll take a closer look at that article soon.

Glad you’re liking dRowAudio, the waveform classes in there are pretty damm fast as they’re tripple buffered to images at various stages which makes drawing them extremely quick. The problem is that they were designed for a relatively small window of sizes so don’t scale to extreme zooming but I never needed them for that (they were originally for a DJ application).

Of course anything I learn will hopefully find a place in the module. I think for this though I’m going to have to go the route of background caching sections of the thumbnail to Images (say three times the visible width) and keep a reserve image ready to render the next block, swapping them over when necessary. Fiddly but the best thing I can think of right now.

Thanks again,
Dave.