Many programs have scrollable waveform views of audio files, which are often zoomable to the point of seeing individual samples.
The obvious way to implement this is using a Viewport of a large Component.
The problem is this: Component's bounds are int-based. When zooming such that each sample is a pixel, at 192kHz, INT_MAX/sampleRate results in 3h 6m. That’s enough to import Interstellar (at 2h 49m) but a slightly longer movie would be out of bounds… If zoom allows a sample to take several pixels we reach the limits sooner.
I wonder what’s a good way to work around this limit, and wonder what JUCE-based DAWs like Waveform do? @dave96
For example in Auto-Align Post we have a waveform viewer with our own custom scrollbar which isn’t based on Viewport (and it’s based on double values counting seconds so this problem doesn’t affect it)
Do people limit the zoom so that the bounds won’t overflow?
Would it make sense to switch Component's bounds to be based on int64?
We have a class that stores the left and right most viewed audio time (usually in seconds). Everything that scrolls listens to it for changes. It has a handy function in for getting the x screen position for a given time in seconds and vice-versa.
We basically do something similar. We have viewLeft and viewRight properties for our arrange window which determines what content to display.
Then the clips are sized to fit in to this arrange Component so they’re never wider than its width. (They can be about 50px outside but that’s really just for drawing rounded corners and decorations etc.)
When the arrange view is scrolled, we update the time that each of the clips is showing then use juce::AudioThumbnail to draw the contents.
You can’t really use a Viewport or full-sized Components for the integer overflow reasons you mentioned and also because this might introduce a lot of computational overhead.
For example, if we scroll so certain clips are off-screen, we simply delete them to free up resources.
I think that’s exactly what we do, though I’ve not bothered limiting the length that a clip could extend to the right/left of the components margins. Maybe we should? Not noticed any problem so far - does it waste CPU to render a 100000 pixel rectangle in a 100px clipped component - I assumed it just clipped it.
The clipping is just the last step in the rendering, before it is handed over to the LowLevelGraphicsContext. All computations including reading the value from the waveform are already executed before.
The handing over to the backend (OpenGL or CoreGraphics, or whatever backend is used) is considered the bottleneck in computer graphics, so the clipping does help, but not rendering the clipped area will help the CPU.
Let me check what we are doing about the waveform rendering That is a good point…
edit: looks like Jules had already saved my bacon there though will need to do the same with our zoom-in render. I found this in AudioThumbnail:const Rectangle<int> clip (g.getClipBounds().getIntersection (area.withWidth (jmin (numSamplesCached, area.getWidth()))));
Yeah, the drawing will get clipped but we ran into issues with the size of the clip Component. Even if it’s not very long, if you view it with a zoom greater than 1 sample/pixel (which is likely , otherwise how do you deal with files of different sample rates?) you can still end up creating clip Components that suffer the integer overflow problem.
This may not happen in reality in a lot of applications, but certainly for a DAW it is likely.
As @dave96 suggested at least from my profiling. it’s best to make sure drawChannel(s) is called on the specific range with high (eg 1px:1sample) zoom levels. because then the cache isn’t enough and you can end up doing A LOT work on your UI thread by the AudioFormatReader not knowing that… (see AudioFormatReader::readMaxLevels())
Ah cool - will look at this. We have all our stuff in memory at the mo and the cache pre-calculated because it isn’t quite a full DAW but … might be a problem in the future
That is a good point…