Basic Gesture Recognition


#1

Just threw together some classes for basic gesture recognition.

Demo: Gesture test
Source: Gesture VC++e project

It’s pretty basic, using a simple moving box boundary crossing technique. A GestureDetector can listen to mouse messages (either globally or on specific components), and will send a GestureEvent to registered GestureListeners when a gesture is completed.

A GestureEvent consists of an array of Directions, and a ModifierKeys. These can be tested for a particular sequence then an action can be performed.

Stuff to do:

  • currently it uses all mouse buttons to trigger the gesture - it should be either right mouse button or an assignable button.

  • resizing triggers gestures too - this is probably fixed by changing it to right-mouse button anyway

  • create a link with ApplicationCommandManager using a GestureListener, allowing the gestures to automatically trigger known commands.

Feel free to do what you want with this. I just wanted to get it to the point where it could fairly reliably generate a series of directions you could test against from a mouse movement.


#2

Blimey, you’re churning out an app almost every day recently!

That’s really cool… I’ll have to have a think about ways of incorporating gestures into my apps.


#3

hehe, i hardly call these ‘apps’ :slight_smile: i’m just making whatever comes into my head.

I was gonna leave this one for a bit, but i think i’ll continue working on it a little longer to make it a bit better. I’m currently doing a ‘GestureMappingSet’ which has obvious purposes.

I’ve made the detector base class abstract so that it’s easier to change the system used to detect the gesture. It’d be cool to have something that stores more interesting data (perhaps a path or something) and then uses clever analysis to process it to generate a gesture. But statistics is the area of maths i avoid, so there’s no chance of me doing that! :hihi: i think the box system works reasonably well anyway.

Anyway, i’ll post an update when it’s done (and it’ll be commented this time :wink: )


#4

Okay, it’s updated (same link as before), and now is easy to link to an ApplicationCommandManager. Also, it’s more heavily commented.

It’s still got plenty of room for improvement, and making the app command link a little smoother.

Here’s a way to test using gestures for app commands…

  • have a BasicGestureDetector and a GestureMappingSet next to your ApplicationCommandManager (be sure to initialise the GestureMappingSet with a pointer to the command manager)

  • call myDetector.listenToMouseGlobally () and myDetector.addListener (&myMappingSet)

  • create gesture configurations and add them to the map.

For example, using the following code…

gestureMap = new GestureMappingSet (&commandManager);

detector.listenToMouseGlobally ();
detector.addListener (gestureMap);

Gesture g;

g.directions.add (Direction::left);	
gestureMap->addGesture (myCommands::foo, g);

g.directions.clear ();
g.directions.add (Direction::right);
g.directions.add (Direction::down);
gestureMap->addGesture (myCommands::bar, g);

Obviously this is a little crude here, but it proves that it works. All it needs is some more simplification functions and stuff.


#5

More updates…

Now the GestureMappingSet can be subclassed to allow for easier assignment of default gestures. A virtual ‘getDefaultGesture (CommandID, gesture&)’ function lets you fill a provided Gesture object with details to map to the given CommandID. Use this function like the getCommandInfo() function in ApplicationCommandTarget. Calling registerDefaultGestures [after the ApplicationCommandManager has registered all the commands] will then apply these gestures, and they will be automatically linked to invoke their mapped commands.

Its packed with even more comments too!

Same link as before (the source project one - the demo app hasn’t changed, as i haven’t been arsed making one that demonstrates them linked to commands… yet)


#6

Hi,
Just bought a new Macbook Pro. The trackpad is multitouch as you probably know and OSX will convert multitouch actions into gestures. Any chance the gesture class you talk about could be updated with support for the OSX gestures. That would be wicked.
I could even give it a try myself if someone could point me out where to start. I know the gestures are NSEvents, but i don’t know how to monitor them. Any ideas?

Cheers
Edwin


#7