In mathematical terms, if P is the maximum detectable period (lowest possible frequency) and p is the frame’s actual period, then the minimum number of samples required to detect the frame’s period using autocorrelation in the time domain is P + p. I’m not sure if this is also possible when using an FFT.
The number of samples required to detect pitch should not be confused with latency though.
They’re similar concepts. There’s basically two ways you can design a pitch detection API:
- Block-based. The detector must always get sent blocks of at least
latencysamples, and will give 1 pitch estimate per block. In this case, the latency would be 2 * the largest detectable period - Sample-based. The detector will accept 1 sample at a time and output an estimate as soon as one is ready. This would allow the detector to take advantage of the
P + pproperty described above, meaning that if the input audio is a higher pitch, you can get that estimate after inputting fewer samples. In this case, the latency of the algorithm is not fixed, it will change with the input pitch.
There is no inherent difference between block and sample based, sample-based is just a special case with a blocksize of one.
I think you confuse block based with FFT based, which has a constant latency, while non-FFT based approaches can take advantage of lower latencies for higher pitches.
Hi everybody, I have just now discovered this repository:
It’s a C library that installs very easily as a JUCE module (just clone it to your user modules directory and add it as usual). The interface is somewhat “unusual” but the library seems overall very useful.
It includes “Helmholz” by Katja Vetter to which I linked above (SNAC/McLeod algorithm) and also “Bitstream Autocorrelation”, by Joel de Guzman. There’s even a mode which combines both. I am getting excellent results with these in my initial tests!
To those who want to try it out, there’s a small mistake in the example: AUDIO_BUFFER_SIZE is wrongly listed among the parameters of LEAF_init(). Also, note that the functions are fed samples one at a time and as soon as they have enough information, the frequency value is updated.
Thanks, at first glance it looks like an interesting combination of methods! It’s probably worth to note that the included FFT claims to violate some patents. Since it’s from the 90s it’s probably nothing to worry about now, but I found it a little irritating given that the LEAF source is licensed as MIT.
Unrelated, but probably interesting too: There seems to be a basic WDF (Wave Digital Filters) implementation in LEAF too.
I wouldn’t say that the included FFT claims to violate some patents. It’s author claimed (back in the 90’s) that the code uses some patented algorithms, gives credit to the authors of those algorithms and claims to be unsure as to how their use might be restricted. But the FFT routine was later included in Pure Data under a BSD-like license. So not only are those patents expired by now, they were permissively licensed anyways.
katjaashome (for anyone that hasn’t seen Katja’s beautiful little corner of the Matrix). Worth 5 mins out of the life of anyone that’s still reading this thread.
8 Likes
Thank you so much for pointing out the beauty of katjaashome, it was not only fun for this old jaded nerd, but it actually got the interest of a shoulder-surfing teenager in the house, too … always a good sign! 
Katja’s dsp page is one of the best. The experiments are gold for anyone learning.
I learned a lot there.
1 Like
Yes, Katja’s home is indeed a very beautiful little corner of the Matrix!
Another algorithm which wasn’t mentioned is Antoine Schmitt’s “Dynamic Wavelet”
(Within HISE there’s a JUCE-like wrapper around it, might save you some time if you want to try it out).
Thanks for the useful info, especially the nUpdate bug notice. I have a new student using this code who will want that fix. A few points:
-
Kalman trackers are also correlation based, but they have the feature of updating the correlation matrix on every sample (unless downsampled, etc.). A simple Kalman tracker under stationary assumptions performs only a few dB better than autocorrelation or power-spectral peak estimation (with the improvement coming from coherent versus non-coherent estimation). It is also easy to update an autocorrelation every sample, so the main difference is the model used in the parameter estimation. Basic autocorrelation methods such as Pitch in SuperCollider look for anything periodic assuming little or no noise. Orchi’s Kalman tracker assumes a sum of N sinusoids, and the model includes their relative phase. (Plain autocorrelation contains no phase information, other than edge effects.)
-
Ignoring the estimate for two maximum-periods makes a lot of sense. In principle, accurate estimates can start after a few samples, assuming no noise or interference, but that’s not reasonable to expect in practice.
-
It is best to adapt the model and covariance-resetting logic to match your signal type and application. The prediction error tells you when the model has “lost lock”, which is ideally only at the beginning and end of a note, but also if a glissando is too fast to track, etc., and the paper discusses this topic a bit. I believe Orchi’s code only covers the specific examples in the paper, so it should be considered only a potential point of departure into Kalman tracking. (I thought the complex formulation was especially cool, and had not seen the Extended Complex Kalman Filter before.) If her code cannot be used to reproduce the examples in the paper, that would be a bug that should be fixed. Otherwise, we should contribute our forks, and ideally coalesce around a common fork that unifies everything we want to add.
2 Likes
Second this, I’ve tested a lot of different algorithms, and MPM is one of the best - very cheap to compute if you do the autocorrelation using the FFT. Add a probabilistic method similar to pyin to improve the tracking stability further for bonus points.
1 Like
Afaik, MPM doesn’t perform well on detecting frequencies below 80Hz. Have you experienced the same?
1 Like
It’s fairly good for me with the human singing voice - though like any algorithm it’s hard to avoid octave errors at points, especially when aiming for low-latency.
It’s all about the peak-picking so you may need to adapt it a bit. I’m currently doing some offline research and using NSDFs with a good peak-picking algorithm can beat PYIN offline if done cleverly.
Some possible ideas:
- In realtime maybe do some octave weighting based on a histogram of previous pitches.
- Once you have a valid pitch track it through time and stick onto it until there’s silence.
- If you get a lot of errors you can try “wisdom of the crowds” and get the median of 3 different pitch detectors:
MPM + FFT frequency picking + Bitstream Autocorrelation (as linked above)
1 Like
I’m recently investigating source separation, so you can remove the transients and increase precision of pitch detection. Should be fairly easy using a median filter