Reading Assignment: Specifying Gestures by Example

Reference Information
Title: Specifying Gestures by Example
Author: Dean Rubine
Citation: "Specifying Gestures by Example", Dean Rubine, SIGGRAPH '91 Proceedings of the 18th anuual conference on Computer graphics and interactive techniques, pp. 329-337, ACM New York, NY, USA, 1991.

Summary
This paper discussed gesture-based interfaces, specifically GRANDMA, which is an object-oriented toolkit for applications with direct manipulation interfaces. It allows for gestures to be added to the interface without being hand coded. A gesture in this sense is a stroke made by a device such as a stylus or a mouse. The gesture recognition toolkit results in a recognizer that is trained from examples of gestures to be able to recognize new gestures that are input to the system.

A gesture-based application, GDP, was described and the GRANDMA toolkit was used to provide gesture recognition for the interface. Gesture classes were used for sets of associated gestures, and are arranged into a hierarchical structure. The GRANDMA toolkit works similar to the Model/View/Controller format, associating an input handler with a view class in order to provide all of its instances and subclasses with access to it.

A limitation of GRANDMA includes the fact that only single stroke gestures are allowed, eliminating the possibility of using more complex symbols. However, it allows for faster recognition, accomplished with a two-phase interaction technique (combining both gestures and the direct manipulation property of the interface) and eager recognition (recognition of unambiguous gestures).  Multi-finger recognition was implemented by processing each finger's stroke as a separate, single stroke then combining them to create a multi-path gesture.

Gesture recognition occurs by first calculating a set of features based off of the various properties (i.e. angles, lengths, etc.) of the gesture, then using each feature to classify the given gesture into one of a set of gesture classes. The classifier is trained using a set of example gestures with an appropriate variance.

The importance of gesture-based interfaces was emphasized multiple times throughout the paper, namely for the ability to improve interactions between humans and computers. It was hoped that this may encourage further integration of gesture recognition in interfaces.

Thoughts
This paper provided a great deal of information regarding gesture recognition. I found it to be very helpful for understanding some of the basic problems and approaches associated with such recognition systems. Published in 1991, this paper strived to encourage further interactions between humans and computers, namely with gesture recognition techniques, including multi-finger touch recognition. This is a topic that is still being emphasized today, although improvements and wider usage has occurred. The system presented, GRANDMA, is an object-oriented system that can apply a hierarchical structure to classes of gestures. The object-oriented nature with the hierarchical structure reminded me of the Sketchpad paper read previously, with its usage of hierarchical structures to easily organize the system and to provide simple extensibility.

An important point of this paper is that it presented a simple, fast gesture recognition algorithm. The extensive use of features for classifying various attributes of a particular gesture distinguished the different properties of a stroke that can be used to compare the differences of various classes of gestures. The use of a classifier for recognizing gestures was simple and easy-to-understand, despite the mathematics associated with calculating the features. The simplicity of this algorithm, combined with its extensibility, provided a foundation for further gesture recognition systems to build upon.