Understanding plugin latency

Hello,

I’d really like to understand how to get to the bottom of the latency issues I’m running into. Let me explain:

I have a VST effect which is able to capture sound from the DAW and sequence it within it’s interface. To trigger it a user presses “Record” which sets the plugin to recording mode and then when the user hits play in their daw I detect the play by checking the “AudioPlayHead::CurrentPositionInfo.isPlaying” flag and it will start copying all the contents of the incoming buffer into it’s own buffer - when the user hits stop than this is then visualized on the screen and the user can hit play etc.

Since it’s a sequencer the amount of tracks/samples and therefore time between the start of processBlock and end of processBlock can vary - I’ve noticed when trying to record in different hosts sometimes the recording is accurate, sometimes its very slightly off, and sometimes noticeably off. I just can’t put my finger on how to adjust this so that no matter the circumstances when I decided to capture the audio from the daw the recorded audio is in sync with the daw. When I hit play in the daw, it should be perfectly aligned with whats recorded.

Any advice appreciated!

It sounds like the issue is not latency, but rather the starting position at playback time. From what I’ve seen, the first sample position is not always the same when restarting. Even if you use Samples as the time for display in the DAW, and explicitly set the start to be at a particular sample, the host may start sending samples earlier than that. So while you started recording at sample 441, say, when you start playback later, it may be start at sample 0 one time, then 441 another time, or pretty much anyplace the host desires. So, you have to check the sample position when playing back, and align your output to the position given. (I’ve even found that in Cubase, the sample position at the start of a processBlock call may be off by one when compared to the previous starting sample position plus the number of samples processed previously!)

1 Like

Ohhhh that’s good to know…I’ve noticed in Cubase the playbackPosition was set to a bizarre small negative amount, but I haven’t been paying attention to the daw sampleCounter…

Thinking out loud here but I’ve also been calculating playback position by calculating timeDifferences but if the user just hit play in the daw I could simply grab the daw position and use that… though I’d have to stick to my own calcs if the user hit play in the vst and didn’t trigger it via the daw…

Thanks for the info!

This value is NOT sample-accurate. It’s ‘block accurate’ i.e. the plugin might observe this change slightly before or after it happened.

Ah, so I’m learning. I’ll experiment with checking isPlaying to detect playback but also adjust my samples accordingly to make sure I’m lined up with the daw.

I’m still having some issues getting this to work properly, and it doesn’t help that I can’t reproduce it and it’s other users complaining about the timing being slightly off. If you look at the screenshot, my vst effect is recording whats coming from the daw. In this example it started at 0, and all of the values were at 0 (position, timeInSamples)

What you guys are saying is that in some cases the daw will report position = 0 but the timeInSamples may NOT line up with that correctly, is that right? In that case, I would need to nudge the final recording back (or forward) by the difference between position and timeInSamples?

How about if a user started recording from the middle of the sequencer, so position 16… I would then need to calculate what the timeInSamples SHOULD be for position 16, check it against the actual timeInSamples, and then nudge it back/forward depending on the value returned. Is this correct?

I’d appreciate any help on this it’s very cryptic to me! Thanks

Hate to bump, but I could use just a bit of extra info if any of you guys can shed some light for me. Again if I could reproduce this issue it would be a bit easier for me to figure out, I guess its soundcard/daw specific

Should I be looking at timeInSamples instead of position?

I’m not sure I get you. Sample position 0 is sample position 0, period. The timeInSamples tells you the sample position of the first sample in the current buffer. If that is not 0 for the first buffer that you receive, then that’s fine, and to be expected (even if you type 0 into the Sample position bar in your DAW). You just need to line up your data with the sample positions given by timeInSamples. So if you need your data to start at time 0, but the first buffer starts at -128 (according to timeInSamples), then you just need to offset your data by 128 samples, so that it is at sample position 0 (because if the buffer starts at -128, then sample position 0 is 128 samples later). Got it?

Oh! According to the documentation “timeInSamples: The current play position, in samples from the start of the timeline.” This doesn’t sound like it’s referring to the current buffer… I’ll have to test it I guess

EDIT: Ok so its definitely the absolute position. So what you’re saying makes sense if someone starts recording from position 0, and it would be trivial to adjust my own data to match. However, if the user is at a different position and then hits record in my plugin, I need to calculate what the timeInSamples would be if there was no offset, and then compare it with what the actual value to determine how much offset I need to account for.

Is that correct? It seems like I could get rounding errors by calculating it myself if I’m not rounding the same way as the host.

Still don’t get you. What do you mean “if the user is at a different position”? You have a buffer. You get timeInSamples. That is the sample position of the first sample in the buffer, regardless of anything else. If it says 32480, then the first sample in that buffer is sample position 32480, period. If you have data that should start at, say, position 1024, then start it at 1024. If the current buffer starts at -128, and there are 512 samples in this buffer, then 1024 is in a later buffer, so you just wait until you get a buffer that position 1024 falls within. In my example, you would see that the next buffer probably starts at -128+512=384, and if it also has 512 samples, then you’d have to wait for the buffer after that, which would likely start at 896, so position 1024 would be offset into that third buffer by 1024-896, or 128 samples into the third buffer. But you cannot guarantee buffers are all the same size, so just check the start of the buffer and the buffer size to see if your desired position is in that buffer, and if so, how far into it.

I also don’t get your phrase “instead of position”. What are you referring to? What is “position”? Except for ppqPosition, which is in quarter notes, what other value exists besides timeInSamples?

All I want to do is record exactly what is coming out of the daw, sample-accurately, the moment the user hits play in their daw. I was told earlier that the isPlaying value is block-accurate but not sample accurate - this would explain why some users are recording sound and it’s off by a small amount. So then how else can I detect playback in a sample accurate way?

I’m not getting how timeInSamples would help in this situation.

I’m sorry if I’m unclear I’m trying my best to explain

You are given the sample position accurately, in timeInSamples. It’s the sample position of the start of the buffer. The comment about isPlaying means you don’t ask if isPlaying is true for any specific sample, you just ask once at the start of processBlock, the same time you ask for timeInSamples. Just once per buffer. Record your data, and memorize the very first sample position you received (once isPlaying is true, which is when you’d actually start recording, I assume, in case you’re in a host where you get procesBlock calls even when isPlaying is not true). When you play back later, if you have made audio corrections or added MIDI or whatever, just line up your data with whatever you’re given at that time. So, for instance, when you recorded, the first timeInSamples started at -128. Ok, so your graph now starts at -128. If you later play back, and the first buffer now says -64, then you have to skip forward 64 samples in your data, from your original -128 to -64. The timeInSamples value is always correct. Just don’t assume your first buffer always starts at that same value. I don’t know how else to explain it.


here you can see 2 quarter notes (black rectangles)
the yellow line implies the project position’s fraction. so basically ppq - floor(ppq).
the green steps indicate the start of the audio blocks that are being sent by the daw. as you can see you only get the ppq at these points and when you look at the point where it transitions from one note to the next you see there is no point on that exact moment where the phase resets, right? so you basically have to take the ppq- and bpm-info and calculate the sample-accurate things from that. you can basically do that with a saw-oscillator that has the same tempo and phase as the project

Howard thanks for your patience and detailed explanations, I feel like I’m so close lol if I could buy you a beer digitally I would! Thanks also Mrugalla!

I think then the issue is the way I’m handling the samples that are recorded. I place them in my sequencer UI based on the ppqPosition, and when the user hits “play” I wait for the ppqPosition to be >= sample position and then start playing the audio by copying it into the outgoing buffer.

However, this can never be sample accurate because what I SHOULD be doing is not use ppqPosition to place my samples on the timeline, but rather keep track of everything based on the timeInSamples. So if it starts a bit earlier or a bit later on the next cycle it doesn’t really matter since I’m just waiting for the timeInSample I get from the processBlock call to be >= to the snapshot I took earlier. If it’s slightly off, then I offset by the amount that it’s off by.

1 Like