resampling the whole buffer is kind of impossible once you start getting into per-sample delay length modulation and any decent sized delay line, source: my first crappy attempt to make a LFO modulated delay. 
For OPs original question, don’t roll your own, just use dsp::DelayLine with pushSample and popSample methods and smooth the delay time parameter, it does exactly what you want.