Video scratching using QuickTimeMovieComponent?


#1

For my DJ app, I’d like to add a “video scratching” feature - that is, the user can, just like scratching audio files using a turntable/cdj (+ timecode vinyl/cd), scratch a video file backwards/forwards using the turntable. Serato does for instance offer this feature. I tried using QuickTimeMovieComponent::setPosition(), but as soon as this function is called it seems to take some time to position the video.

How can I achieve this?

PS: I used JUCE147’s QuickTimeMovieComponent together with the latest QuickTime SDK on Windows.


#2

Well, QT gives you a speed control, but exactly how well it’ll play video is completely unpredictable. It might be ok if your movie has no i-frames, but most compressed video formats really aren’t designed to play backwards (or even at speeds other than 1x)


#3

yes, I tried using setSpeed(2.0) - that works fine, but as soon as I used setSpeed(-2.0) it starts creating “jumps/skips” in the video.

Do you have any idea how other apps achieve the ability to scratch videos?

PS: I got said that Max/MSP uses Quicktime - and as far as I know there are apps in Max/MSP that allow video scratching (Ms. Pinky) - so might there be a setting in Quicktime for this purpose?


#4

I think it really just depends on your format, it’s the i-frames that kill it.

If you were writing your own player, you could use a huge buffer of uncompressed frames, which would help, but if you’re using QT, you’re really at their mercy.


#5

… on a related note would be possible/sensible to separate the QuickTimeComponent into a Component and a Movie “peer” of some sort. This would be similar to the way the CameraDevice stuff works.

Again this wold be similar to the way Max/MSP/Jitter allows access to frames to process etc. E.g. using something like a “QTMovieImageListener” to get Images.


#6

[quote=“jules”]I think it really just depends on your format, it’s the i-frames that kill it.

If you were writing your own player, you could use a huge buffer of uncompressed frames, which would help, but if you’re using QT, you’re really at their mercy.[/quote]
Yeah that’s the idea i had too. You only need like 4 seconds of video buffered or so (I know that this quickly amounts to nearly 100MB). Would it be hard to change the QuickTimeMovieComponent to internally buffer frames before displaying them?

Example: User sets playback position to 0min 20sec, so the code buffers from 0min 16sec on (4 seconds) up to 0min 20sec. That’s 100 uncompressed images if the framerate is 25fps.


#7

[quote=“martinrobinson”]… on a related note would be possible/sensible to separate the QuickTimeComponent into a Component and a Movie “peer” of some sort. This would be similar to the way the CameraDevice stuff works.

Again this wold be similar to the way Max/MSP/Jitter allows access to frames to process etc. E.g. using something like a “QTMovieImageListener” to get Images.[/quote]

well… it’d be sort of possible to break it into a movie and a component, though getting images probably wouldn’t be possible in realtime, because QT is fairly opaque about the pipeline between file and the screen. It’d be quite a lot of work to separate it though, because it’s all different on mac/pc now, so I’m not too enthusiastic about doing that!

As far as I know, it’d be impossible - with QT you just give it a movie and tell it to “play”, you don’t get any control or access to how it actually handles the decompression…


#8

Anyway I am now confident that it definitely is possible to do video scratching via Quicktime, as Serato uses Quicktime for that purpose. Quoted from a website “Serato Video Scratch uses Apple’s Quicktime technology to playback videos. This means that we will be able to support a wide range of formats”.