Signal-Driven Visualizations for Music


#1

I plan to use JUCE to create visualizations for music, because I have always been disappointed with the visualizations for music in Windows Media Player. This is because I prefer visualizations for music that are entirely driven by the music.

These visualizations will be for two-channel recordings. The left channel signal will be sent to the X axis of an oscilloscope, and the right channel signal will be sent to the Y axis of the oscilloscope. This is known as Quadrature Art.

When this is done with sine waves of different frequencies or phase angles, the result may be Lissajous curves. To see an animation of Lissajous curves, go to Wikipedia.

Another aspect of the visualizations will be to use the aperiodic music signal to simultaneously control the amplitude and frequency of periodic oscillators. It seems to me that this would produce some very strange results.

The FFT data about the music could be divided into three frequency bands. The bass could generate blue imagery, the midrange could generate green imagery, and the treble could generate red imagery.

Various mathematical operations could be applied to the FFT data derived from the music. Phase Vocoder analysis consists of taking one FFT reading, and calculating the reading’s absolute value, as well as the square of the absolute value. What happens when one does Quadrature Art with absolute values?

@ danielrudrich
I have seen Goniometer displays before, but I did not know them by name, nor was I aware of how many kinds of information they provide. Goniometers provide information that I have wanted to access for a long time. The Goniometer will make it easier to time-align the left and right channels of two-channel recordings. This will make me happy, because all of the audio CDs in my music collection have small timing offsets between their left and right channels that vary from CD to CD.

@ daniel
One of the benefits of using FFT is the option of using Phase Vocoder analysis to affect the visualizations. I do not know what this would look like, but I am intrigued by the idea of using absolute values with a Goniometer.


#2

That’s basically what a Goniometer does, which is used for checking the left-right correlation in a stereo mix.


Maybe there are already multiband-goniometers out there.


#3

IMHO it makes no sense to use FFT in this case, since it is more known to introduce artefacts, rather than analysing with the accuracy that would be needed to get anything useful of that visualisation.
But I am only looking at it from a high level perspective.


#4

Agreed if you want colourisation from frequency separation it may be wiser just looking at the difference between any to given samples. For the most part higher frequencies will have greater distance between them than lower ones. Obviously extremes in volume would make this a less accurate.

If wanted to go a little deeper you could just measure role off of a single pole eq and difference that against the original signal and that would mitigate extreme volume changes and still be very efficient for visualisation.