I’m implementing a waveform from an audio file that can be zoomed up to a sample level. Performance wise the path rendering isn’t a problem, but the data processing might be: the bigger the file, the more samples must be evaluated. Downsampling (discarding samples to make it faster) creates aliasing when zooming, so I was thinking about lower resolution representation of the min/max samples (i.e for each 32 samples find the min/max and use that to draw), but for big files like 15-30 minutes this seems like a lot of samples to make it work smoothly on each zoom iteration. Doing it with preacalculated mipmaps would take a good amount of memory aswell and zoom wouldn’t be that smooth. Maybe an hybrid method between those two?
I wonder how DAWs like Ableton are doing this smoothly and in a responsive way without sweating, even with huge files.
i recently wrote something like that too and my approach rn is to only read as many samples as there are pixels on the x-achsis, no matter the zoom level or position. it still needs some improvements, like what you said about finding the max of each sample index. maybe it would be sufficient to just read a few sorrounding samples to approx max instead of finding the actual max
IIRC, Ableton produces .asd files which contain amongst other stuff peak data specifically for speeding up rendering. Reaper does a similar thing. Not sure about the format so you’ll have to do a bit of research.
I actually tried to look into the .asd files to get a grasp on how many samples per file they take but it’s all on binary data.
Maybe an hybrid between what you both said? Having kind of an .asd file or aux buffer with the analyzed waveform storing all its peaks (min and max values) and if that’s still bigger than X samples per zoom, use a downsampled version of that reading every 8-16-32 peaks.
As a curiosity, I loaded a 37 min 410mb file in Ableton and it generated a 2mb .asd file, the memory usage only increased 20mb aprox (playback may be streaming from disk using a background thread instead of storing all file in a buffer) and the CPU usage didn’t change compared to zooming in/out the Ableton UI (in arrangement view). So it barely uses the CPU to process the waveform data when zooming.
@johngalt91 Any possibility you would share some code of your custom waveformview class? I’ve been using AudioThumbnail untill now and I can zoom in and everything perfectly well, but I’m not pleased with how it looks. I think the RectangleList looks quite ruff easily. and when zoomed to the sample level it becomes a sample-and-hold type of ugly signal. When I look at the way ableton does it, it looks perfect zoomed totally out and totally in. I’m looking for alternative’s to AudioThumbnail but did not find any yet.
I just wrote the temporary solution of a fixed waveform without the zooming part and put that on standby as that was taking me time and I needed to advance other stuff.
I have yet to revisit the code but I found that I had to do trilinear interpolation (between 2 samples in 2 different mipmaps, and then another interpolation between these). Maybe there’s a more efficient way and someone else that has this already figured out can give us some insight
I actually discovered your modification already - and implemented it! It’s awesome! Thnx for that.
I think overall I would love something more like the way it is done in ableton where dots representing the samples are connected with a non-linear connecting line when zoomed-in. And on the other extreme -completely zoomed-out- the waves look like they are smoothed or averaged or something… using the AudioThumbnail it looks really rough.
The zoomed out audiothumbnail uses rectangles, that’s why it looks so sharp (Ableton uses paths), and the zoomed in with @oxxyyd mod uses paths but from point to point (lines), while Ableton probably uses splines with cubic/Hermite interpolation.
The problem with the zoomed out version is if you use paths there will be a lot of edges in the edgetable (basically 1-2 per pixel?) and it gets painfully slow. So how Ableton does it so efficiently is the real deal
Yeah I don’t understand it either…
Is there some kind of way to maybe apply a sort of anti-aliasing process on the rectangleList which could make it look more smooth?
I guess not, haha.
If the data in the view does not change too often you could look into using the DSP techniques of upsampling when you are zooming in, and downsampling when zooming out. you will still have to do some interpolation to get points at pixels, but you could get away with just plotting nearest values if accuracy is not of importance.
I’d say the difference in displaying the thumbnail is weather you want to display the samples or the waveform resulting from the samples. The last ableton waveform displays a lot of “waviness” just before the steps. But the samples (as displayed by the juce waveform) are rather uniform (same values) before the step.
With the last juce thumbnail you could rather easily cut it just before the last step, time reverse it and paste it to the right of the original wave just before the step, creating a mirror of the waveform just before the step, thus creating a continuous waveform. With the ableton implementation I think it would be more difficult to hit the right spot to do that cut.
By the way, if you cut a real-world waveform in Ableton, what does the resulting waveform look like at the cut? E.g. if you cut the last waveform right at the zero crossing of the last step, what will it look like?
Untill macOS Sonoma I had amazing performance using paths to draw dense waveforms, then some bug broke the path performance making it completely unusable. Never saw that get resolved. It used to be buttery smooth zooming for me. I got forced to switch from beautiful paths to uglier rectangles.
Maybe it could be a good idea to use a different renderer than the OS one for this?
I don’t care at all about accuracy. I am aware that the juce version is actually the more accurate one since it doesn’t do “imaginary” interpolation of the samples like ableton is doing. I guess once the digital signal goes through a DAC it will result in such a signal though…?
Anyway, as I said, I just want to make my juce gui a bit more pleasing to the eye!
