Warp example?

[erikronstrom]

Hi,

Is there any example of how to use warp markers?

I’m trying to insert some markers into a clip but that causes the audio file to start from the beginning every time a marker is hit, rather than the audio being stretched.

This is my attempt:

    auto clip = EngineHelpers::loadAudioFileAsClip(*track, file, pos);
        
    clip->setWarpTime(true);
    clip->setAutoPitch (false);
    clip->setAutoTempo (false);
    clip->setTimeStretchMode (te::TimeStretcher::defaultMode);
    auto& wtm = clip->getWarpTimeManager();
    wtm.insertMarker(te::WarpMarker(1.0, 1.0));
    wtm.insertMarker(te::WarpMarker(2.0, 2.5));
    wtm.insertMarker(te::WarpMarker(3.0, 3.0));
    wtm.insertMarker(te::WarpMarker(4.0, 4.5));
    wtm.insertMarker(te::WarpMarker(5.0, 5.0));
    clip->updateSourceFile();

Erik

[erikronstrom]

Anything? I would really like to do some warping!

[dave96]

Two things spring initially to mind:

1. Do you have a timestretch algorithm enabled by way of config flags? Current options are Elastique ($) or the built-in SoundTouch (more on this in the near future…)
2. Are you trying to do this synchronously? The warping will trigger a file render that needs time to render and then update the play graph. Are you giving this time to happen?

[erikronstrom]

Do you have a timestretch algorithm enabled by way of config flags?

Yes, SoundTouch is enabled. (At first, before I had any timestretch algorithm enabled, the rendered audio files contained really strange garbage )

Are you trying to do this synchronously?

No, I just load a file into a clip and place warp markers and later start playback. The audio that is played back matches the content of the rendered audio file (i.e. the content of the rendered audio file is also bad).

[dave96]

As far as I can tell what you’ve got is correct.
I’ve just tested this in Waveform and end up with the following XML:

      <WARPTIME warpEndMarkerTime="4.234916666666667" timecodeFormat="beats"
                viewleft="0.0" viewright="4.234916666666667">
        <WARPMARKERS>
          <WARPMARKER sourceTime="0.0" warpTime="0.0"/>
          <WARPMARKER sourceTime="0.533062496687906" warpTime="1.5625"/>
          <WARPMARKER sourceTime="4.234916666666667" warpTime="4.234916666666667"/>
        </WARPMARKERS>
      </WARPTIME>

I presume yours is similar? The only marker I added explicitly was the middle one. I think the end ones get added automatically.

The only thing I can think of is that SoundTouch isn’t behaving itself. I’ve actually refactored this code over the past couple of days which might improve SoundTouch performance, I just need to get the final tweaks finished up…

[erikronstrom]

Yes, my clip looks similar:

<AUDIOCLIP name="c" start="4.0" length="4.969070294784579" offset="0.0"
           id="1080" source="/Users/admin/Music/c.wav" sync="0" elastiqueMode="4"
           pan="0.0" colour="ffffaa00" warpTime="1" autoPitch="0" autoTempo="0">
  <LOOPINFO rootNote="-1" numBeats="4.96907029478458" oneShot="0" denominator="4"
            numerator="4" bpm="0.0" inMarker="0" outMarker="-1"/>
  <WARPTIME warpEndMarkerTime="4.96907029478458">
    <WARPMARKERS>
      <WARPMARKER sourceTime="0.0" warpTime="0.0"/>
      <WARPMARKER sourceTime="1.0" warpTime="1.0"/>
      <WARPMARKER sourceTime="1.5" warpTime="2.0"/>
      <WARPMARKER sourceTime="3.0" warpTime="3.0"/>
      <WARPMARKER sourceTime="3.5" warpTime="4.0"/>
      <WARPMARKER sourceTime="4.96907029478458" warpTime="4.96907029478458"/>
    </WARPMARKERS>
  </WARPTIME>
</AUDIOCLIP>

[dave96]

Is there any chance you could make this in to a PIP like the example projects so I can test it quickly?

[splintersilk]

Hi @erikronstrom and @dave96
Might it be possible to get a bit more information on what’s happening here please?
I tried following @erikronstrom’s steps but when I get to the clip->updateSourceFile() I get silence on playback. I’m a bit confused as to what updateSourceFile() does as to me it implies that it destructively modifies the source audio played by the clip. That can’t be right/I must be confused!

How do we define the temp audio file that gets written when the warptime gets rendered? Or is that not the intention of how this is supposed to work?

Many thanks in advance!
Jeff

[dave96]

@splintersilk

I’m a bit confused as to what updateSourceFile() does as to me it implies that it destructively modifies the source audio played by the clip. That can’t be right/I must be confused!

That is correct unless you call setUsesProxy (false) on your clip. If you do that, the warp will happen dynamically.

[splintersilk]

Thanks @dave96! Much appreciate the fast response.

I’ve actually tried it both ways.

//acb is clip via track->getClips(), dynamic cast to an audioClipBase
acb->setUsesProxy(true);
acb->setWarpTime(true);
acb->setAutoPitch (false);
acb->setAutoTempo (false);
acb->setTimeStretchMode (t::TimeStretcher::rubberbandMelodic); //or elastiqueMobile or soundtouchNormal
auto& wtm = acb->getWarpTimeManager();
DBG(wtm.getSourceLength().inSeconds());
wtm.removeAllMarkers();
wtm.insertMarker(te::WarpMarker(3.1s, 1.1s));
wtm.insertMarker(te::WarpMarker(3.2s, 2.1s));
acb->updateSourceFile();
DBG(wtm.getSourceFile().getFile().getFullPathName());

That gives an edit file of:

    <AUDIOCLIP name="Track 1 Recording 1" start="39.65475082397461" length="3.304562500000003"
               offset="0.0" id="1063" source="_Track 1_Take_1.wav" sync="0"
               elastiqueMode="10" pan="0.0" colour="ffff0000" proxyAllowed="1"
               loopLength="3.304562568664551" loopStart="0.0"
               mute="0" gain="16.7320384979248" warpTime="1"
               autoPitch="0" autoTempo="0">
      <LOOPINFO rootNote="-1" numBeats="3.304562500000003" oneShot="0" denominator="4"
                numerator="4" bpm="0.0" inMarker="0" outMarker="-1"/>
      <WARPTIME warpEndMarkerTime="3.3045625">
        <WARPMARKERS>
          <WARPMARKER sourceTime="0.0" warpTime="0.0"/>
          <WARPMARKER sourceTime="3.1" warpTime="1.1"/>
          <WARPMARKER sourceTime="3.2" warpTime="2.1"/>
          <WARPMARKER sourceTime="3.3045625" warpTime="3.3045625"/>
        </WARPMARKERS>
      </WARPTIME>
    </AUDIOCLIP>

With the setUsesProxy(true) as above, there seems to be no change to the file on disk. I used a DBG to see if it was actually writing a temp file somewhere else but it wasn’t as far as I could see. Much like @erikronstrom I get the audio re-starting each time a marker is hit.

setUsesProxy(false) and removing updateSourceFile() resulted in the unaltered source audio playing back.

I’m definitely missing something (or multiple things), but I can’t figure out what.

I’m using 2d566dcd1b6 from Dec 21, 2022, which is a little old. I’m going to see about grabbing the latest Develop commit later today.

[splintersilk]

Alas, same behaviour with 9434651d66. Hmm I’m not sure what I’m doing wrong, but I either get no change, or the clip restarting at each warpMarker per what @erikronstrom mentioned.

[splintersilk]

I thought I’d take a look at tracktion Waveform Free to see if that gave me any clues. I couldn’t see how to export an XML unfortunately but I did note that Waveform uses a WarpTime clip effect. Does a beautiful job of the warping too btw. Anyway tried using a clip effect as welZ again nothing promising yet.

[dave96]

@splintersilk

I couldn’t see how to export an XML

You don’t need to export anything, it’s XML already. Just open the .tracktionedit file in a text editor.

Looking at the code, you might need to set setAutoTempo (true) to get the real-time warping. That might be an oversight on our part though. There’s a lot of things going on in WaveNodeRealTime.

Put a breakpoint in WaveNodeRealTime::buildAudioReaderGraph and see if it stops under the if (warpMap) statement.

[Fernando]

Hi Dave,
I’m also having problems with warp markers.
I figured out how to create/destroy and change the warp time of a warp marker and it seems that the engine underneath reacts well.
My settings are.
WarpTime = true
AutoPitch & AutoTime = false
proxyAllowed = true
I find myself with two issues :

1. I am in “proxy” mode and when I change the position of a warp marker it triggers the assert in the “deleteFile”.
   
   
   
   
   
   
   Here is my code to change the markerPosition :

void mouseDrag(const MouseEvent& e) override {

				beginDragAutoRepeat(10);

				auto posX = e.position.x;

				if (index == -1) return;

				auto newWarpTime = TimePosition::fromSeconds(evs.xToTime(posX, getParentWidth(), evs.editorViewX1, evs.editorViewX2, false));

				if (wac->getWarpTimeManager().getMarkers()[index]->warpTime == newWarpTime) return;

				wac->getWarpTimeManager().moveMarker(index, newWarpTime);

			}

2. The waveform doesn’t seem to redraw well… Should i need some listener/callback to update my thumbnail when the renderJob is completed ?

In this regard I was wondering if I could get some advice and in general what should be the workflow to follow.

Thank you.

[dave96]

I recommend using:

WarpTime = true
AutoPitch = ?
AutoTempo = true
proxyAllowed = false

Then you don’t have to deal with the proxy file.

I image it’s failing to delete the file in your code because you’ve still got it open for drawing somewhere? Windows is a pain like that.

[Fernando]

Thank you for your response,
when the proxy is disabled the thumbnail seems not to update.
After moving the marker do I have to perform some kind of special operation to make the update ?

Here is my code to create/update the thumbnail.

void AudioClipComponent::updateThumbnail()
		{
			if (auto* wac = getWaveAudioClip())
			{
				te::AudioFile af(wac->getAudioFile());

				if (af.getFile().existsAsFile() || (!wac->usesSourceFile()))
				{
					if (af.isValid())
					{
						const te::AudioFile proxy((wac->hasAnyTakes() && wac->isShowingTakes()) ? wac->getAudioFile() : wac->getPlaybackFile());

						if (thumbnail == nullptr) {
							thumbnail = std::make_unique<te::SmartThumbnail>(wac->edit.engine, proxy, *this, &wac->edit);
						}
						else
							thumbnail->setNewFile(proxy);
					}
					else
					{
						thumbnail = nullptr;
					}
				}
			}

			repaint();

		}

In the paint i always check if thumb.isOutOfDate() and if it’s return true i call the updateThumbnail().

I really appreciate the way tracktion draws the waveform ( particularly in the wave audio editor with the real-time warping ) and want to reach something similiar.

[splintersilk]

@dave96

You don’t need to export anything, it’s XML already. Just open the .tracktionedit file in a text editor.

Ah yes. I saw that file but my Mac was reporting it as 0 byte length. On saving refreshing etc it is indeed the XML I was hoping for - thanks for that!

Copying and pasting the XML from that into my app/edit got the same audible result which was a delight! However I did notice that Waveform has all of the warping stuff under EFFECTS>EFFECT Type=“warpTime”>WARPTIME etc. The code I was testing to this point was setting up the WARPTIME block as a sibling to the LOOPINFO block, with no mention of the EFFECTS block at all - and indeed didn’t work.

Which is the right approach? Are they both correct?

[splintersilk]

@dave96

Looking at the code, you might need to set setAutoTempo (true) to get the real-time warping. That might be an oversight on our part though. There’s a lot of things going on in WaveNodeRealTime.

Put a breakpoint in WaveNodeRealTime::buildAudioReaderGraph and see if it stops under the if (warpMap) statement.

Thanks for this suggestions. To confirm, setAutoTempo(true) seems mandatory for warping. Now it works well and without messing with clip effects. Additionally the ‘repeating from start’ issue seems isolated to using setUsesProxy(true).

However I would prefer not have to use autoTempo as it seems to interfere with loopStart. You hinted that it might be possible to get rid of that dependency. Is that something I might be able to test at my end?