Hi all,
I’m building Go.dot, a GPL-3 theatre playback application on Tracktion Engine. Every cue is a clip in a launcher slot. While adding a per-clip playback speed (0 to 20×, varispeed or timestretch, changing while the clip plays), we ran into a few things in `tracktion_WaveNode.cpp` and `tracktion_SlotControlNode.cpp`. They aren’t specific to our use: they affect launcher clips and auto-tempo clips at any ratio other than 1, including a file at a different sample rate from the device.
Line numbers are in `tracktion_WaveNode.cpp` unless another file is named, at `13b5132` (23 September). We checked `develop` at `cf5cd87` today, and none of these files has changed since. Our changes are a patch series in our repository, with every hunk commented:
Unless stated otherwise, the measurements are on launched clips at 48 kHz, with Signalsmith as the stretcher.
### 1. `LagrangeResamplerReader` resamples each block at a rounded ratio and moves the file every block
- `:329` reads `round (numDestFrames * ratio)` source frames, and `:376` resamples at *that count ÷ numDestFrames*, not at `ratio`.
- `:298` moves the file to the requested position on every block. The reset at `:295` only covers jumps of more than a sample, and the interpolator’s own consumption (`processAdding`'s return value) is ignored. So the file position and the interpolator drift apart.
- Whenever `numDestFrames * ratio` isn’t a whole number, each block plays at a slightly wrong ratio, and a frame is skipped or repeated at the start of some blocks. For example, a 44.1 kHz file on a 48 kHz device in blocks of 256 needs 235.2 frames a block. It reads 235 every block while the requested position advances by 235.2, so one frame is skipped every fifth block.
- We measured the click on a speed ramp: a 1 kHz sine at 0.25 amplitude, whose own largest sample-to-sample step is 0.033, showed steps of 0.045 at block starts. The steady 44.1-on-48 case comes from the same lines; we didn’t measure it separately.
- The warp path does the same rounding (`:1498`, `:1504`).
- Our fix passes the exact ratio and puts back the frames the interpolator didn’t consume. `setPosition` only moves the file on a real jump, and `getPosition` returns the exact position the reader has reached rather than the file’s whole-frame position.
### 2. The stretcher is primed as if input and output frames were the same thing
- `TimeStretchReader::primeStretcher` (`:736-767`, and the same in `ReadAheadTimeStretchReader` at `:913-925`) backs up `latencySamples` input frames, then discards `(position - backupPos) + latencySamples` *output* frames. That only holds at speed 1. At 20× the prime reads about 20 times too much input and starts in the wrong place.
- Signalsmith’s delay, counted in input samples, isn’t a fixed number either. It is `inputLatency() + speed * outputLatency()`, which is its own `outputSeekLength`. `TimeStretcher::getLatencySamples()` returns the plain sum, so a stretcher primed at speed 1 and then played at speed *s* is `(1 - s) * outputLatency` out of place in its source.
- Measured: the sound was 95 ms early at 0.5× and 47 ms late at 2×, in heard time.
- Our fix adds `TimeStretcher::getOutputLatencySamples()`. Signalsmith returns `stretch.outputLatency()`; the others return 0, which keeps the current behaviour. The prime then runs at the speed the clip plays at: it backs up `(latency - outLatency) + s * outLatency` input samples and discards that distance ÷ *s* output frames.
### 3. A stretcher built before its file is in the cache primes on silence
- `TimeStretchReader` primes in its constructor (`:613`), which runs when the graph is built. A real-time `AudioFileCacheReader` read has a 0 ms timeout. For a file the cache has only just been given, it fails, `feedAndProcess` returns false (`:719`) and the prime loop stops. The stretcher is left primed on nothing.
- Measured: every stretched clip whose graph was built before its file was mapped (for us, every newly armed clip) opened with a whole stretcher latency of silence (about 150 ms at 48 kHz), and the first moments of its file were lost.
- Our fix has `primeStretcher` report whether it got all the way. If it didn’t, the first `readSamples` (the launch) primes again.
### 4. A cache miss during playback makes a stretched clip fall behind
- `feedAndProcess` returns false when a source read fails (`:719`), and `readSamples` gives up the rest of the block. A real-time cache read also fails whenever the cache’s own thread holds the reader, so this happens under load, not only at start-up.
- The resampler path keeps time through a miss. The stretch path keeps its old place, so everything after the miss plays late. On GitHub’s shared CI machines we saw stretched clips 10 to 60 ms late.
- Our fix feeds the stretcher silence for input the cache couldn’t hand over, and carries on. Only the prime at construction stays strict.
### 5. A slot rebuilt while its clip plays tells the clip the playhead jumped
- `SlotControlNode::prepareToPlay` (`tracktion_SlotControlNode.cpp:92-97`) takes `lastSamples` from the node it replaces, but not `wasPlaying`. So the new node’s first block goes through the `! wasPlaying` branch (`tracktion_SlotControlNode.cpp:249`), sets `playheadJumped`, and the clip’s `TimeRangeReader` resets its readers (`:1167`).
- At a ratio of exactly 1 this is inaudible. At any other ratio the Lagrange interpolator loses its sub-sample position, which gives a faint tick in every playing launched clip at every graph rebuild. For us that is every time another cue is armed. (A stretched clip is spared: `TimeStretchReader::reset()` is empty.)
- Our fix carries `wasPlaying` over from the replaced node.
### 6. A beat-based clip’s loop restarts the stretcher at every pass
- `BeatRangeReader::readLoopedBeatRange` splits a block at the loop end and reads the second part from the loop start (`:1336`). To the reader below, that is a jump back, and `TimeStretchReader::setPosition` resets the stretcher and primes it again on any jump of more than 10 samples (`:621-634`). So the stretcher starts again at every pass.
- Time-based clips don’t do this, because their loop is set on the `AudioFileCacheReader`, below the stretcher. Auto-tempo and launcher clips loop above it.
- Measured on a looped 1 s at 1×: 1.5 to 4 ms of priming on the audio thread in the block of the wrap, then a gap of 256 samples and a click. The gap turned out to be `DeviceManager`'s CPU-overload mute (`tracktion_DeviceManager.cpp:1376`) answering the long priming block. Since the muted block isn’t processed, everything after it also plays a block late.
- Our fix, for launched clips of a single tempo: a small reader just above the file-cache reader. It counts position on past the loop end and wraps only where it reads the file, keeping the loop’s exact fractional length and starting every pass on the loop’s first frame. `BeatRangeReader` then passes the beats on unwrapped. The resampler and the stretcher now read one unbroken stream. A wrap costs an ordinary block (35 to 61 µs), and the gap and click are gone.
### Two smaller ones
- **Auto-tempo clips always get the default stretcher.** In `AudioClipBase::getActualTimeStretchMode` (`tracktion_AudioClipBase.cpp:669`), once a stretcher is compiled in, every launcher clip left at `disabled` is stretched anyway, even at 1:1: it stops being sample-exact and costs a stretcher’s CPU. We added a virtual, `EngineBehaviour::autoTempoClipsUseDefaultTimeStretcher()`, which defaults to `true` (the current behaviour), so a host can keep a clip on the resampler. It’s about ten lines, in `0001-auto-tempo-clips-may-resample.patch` next to the one above.
- **Very large reads.** At high ratios, `LagrangeResamplerReader` can ask for more frames than an `AudioScratchBuffer` holds without growing, or more than the file cache allows in one read (48,000 frames). We split such reads in two. This only happens at extreme speeds.
The patches are GPL-3 in our repository. If any of this is more useful to you as code than as a report, let me know how you’d like to receive it.
Thanks for the engine. It has carried a lot of this project.
Pierre-Olivier