Reference Information
Title: Protractor: A Fast and Accurate Gesture Recognizer
Author: Yang Li
Citation: "Protractor: A Fast and Accurate Gesture Recognizer", Yang Li, Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 2169-2172, 2010.
Summary
This paper discussed Protractor, which is a template-based gesture recognizer that uses nearest neighbor for classifying new gestures based on a set of pre-classified training gestures. The nearest neighbor approach works by comparing an unknown gesture at runtime to the training examples, and classifying the unknown one as that of the class of its K nearest neighbors in terms of similarity. The method used for comparing the similarity of gestures with Protractor is claimed to be novel, using minimum angular distance to calculate similarity. This contributes to the recognizer's accuracy, speed, and minimization of memory usage. It was suggested that these features of Protractor make it a good choice for use on mobile devices, where memory and processing power are in limited supply. The usage of nearest neighbors also allows for personalization by users. Preprocessing of gestures was performed before comparison, involving resampling and noise reduction similar to that of the $1 recognizer. One of the more unique features of Protractor is its ability to recognize variation with regards to the orientation of a gesture.
Protractor was compared to the $1 recognizer during evaluation, and a similar experiment was conducted using the same data that the $1 recognizer was tested on in its comparisons to other recognizers. The results of the experiment showed that Protractor performs faster, no less accurately, and uses less memory than the $1 recognizer. In addition, the experiment was performed on a mobile device to analyze its usefulness for mobile applications.
Thoughts
The main concern that I had while reading about Protractor is the slowness that generally comes with the usage of nearest neighbors algorithms. Since the algorithm requires comparisons at runtime to each of the known gestures, the runtime computations could be costly.Therefore, I was rather surprised that the experiments run on a mobile device showed it to be a feasible solution for mobile applications. I thought that this was a great test to perform, as memory and time constraints are something that must be considered for mobile platforms.
In addition, the ability to use a nearest neighbors recognizer that is both fast and that requires smaller amounts of memory is beneficial due to the amount of personalization that can be provided by its use. Since the nearest neighbors algorithm simply needs to compare gestures based on their associated data, it is much simpler for gestures to be added to the database, allowing for more opportunities for user-customization. This attribute of Protractor could allow it to be applied to more applications. Combined with its ability to run reasonably on mobile devices, this opens up even more possibilities for usage of the Protractor recognizer.
Thursday, February 28, 2013
Thursday, February 14, 2013
Reading Assignment: PaleoSketch: Accurate Primitive Sketch Recognition and Beautification
Reference Information
Title: PaleoSketch: Accurate Primitive Sketch Recognition and Beautification
Authors: Brandon Paulson and Tracy Hammond
Citation: "PaleoSketch: Accurate Primitive Sketch Recognition and Beautification", Brandon Paulson and Tracy Hammond, Proceedings of the 13th International Conference on Intelligent User Interfaces, pp. 1-10, 2008.
Summary
This paper discussed a recognizer of low-level, primitive gestures that produces beautified versions of the gestures. The motivation behind the creation of this recognizer was to provide a means for integrating sketch recognition into user interfaces for freely-drawn sketches. The idea of the recognizer is to be able to recognize primitive gestures that then provide a foundation for creating more complex shapes by combining primitive shapes hierarchically. In order to improve upon other sketch recognition algorithms, two new features were added (NDDE and DCR) and a new ranking algorithm was used.
The recognizer works by taking a single stroke, calculating normalized distance between extremes (NDDE) and direction change ratio (DCR), then sending the data through a series of recognizers for each primitive that the system is designed to recognize (line, polyline, ellipse, circle, arc, curve, spiral, helix, and complex). The results of each recognizer are sorted into a hierarchy and ranked.
Experiments were conducted to collect drawn shapes from users, train the system on that data, and then test it against more data collected from users. The data was tested on both the recognizer described in this paper and other notable recognizers. It was shown that the recognizer has improved accuracy of recognition as compared to the other algorithms and that it also recognizes more primitives. Accuracy is most notably improved with regards to recognizing polylines and curves.
Thoughts
A motivation of this work that was discussed, providing an easier means of integrating sketch recognition into user interfaces, is very similar to that of the quill system that we read about in a previous reading assignment. Other topics that were mentioned in this paper that we have read about in previous assignments included the Sketchpad, Rubine, and Long work that were all cited as previous work that influenced the recognizer discussed within this paper. The previous reading assignment regarded a hybrid recognizer that was mentioned as a future goal within this paper.
I found the fact that a gesture is run through multiple recognizers, one for each primitive shape that the system is capable of recognizing, to be very interesting. Since the results of each are ranked, this would be useful for when a particular gesture is similar to multiple types of shapes, since each shape's likelihood of recognition would then be ranked. Also, the idea of building up complex shapes from a series of primitives seems like a very useful process for recognizing complex gestures.
Title: PaleoSketch: Accurate Primitive Sketch Recognition and Beautification
Authors: Brandon Paulson and Tracy Hammond
Citation: "PaleoSketch: Accurate Primitive Sketch Recognition and Beautification", Brandon Paulson and Tracy Hammond, Proceedings of the 13th International Conference on Intelligent User Interfaces, pp. 1-10, 2008.
Summary
This paper discussed a recognizer of low-level, primitive gestures that produces beautified versions of the gestures. The motivation behind the creation of this recognizer was to provide a means for integrating sketch recognition into user interfaces for freely-drawn sketches. The idea of the recognizer is to be able to recognize primitive gestures that then provide a foundation for creating more complex shapes by combining primitive shapes hierarchically. In order to improve upon other sketch recognition algorithms, two new features were added (NDDE and DCR) and a new ranking algorithm was used.
The recognizer works by taking a single stroke, calculating normalized distance between extremes (NDDE) and direction change ratio (DCR), then sending the data through a series of recognizers for each primitive that the system is designed to recognize (line, polyline, ellipse, circle, arc, curve, spiral, helix, and complex). The results of each recognizer are sorted into a hierarchy and ranked.
Experiments were conducted to collect drawn shapes from users, train the system on that data, and then test it against more data collected from users. The data was tested on both the recognizer described in this paper and other notable recognizers. It was shown that the recognizer has improved accuracy of recognition as compared to the other algorithms and that it also recognizes more primitives. Accuracy is most notably improved with regards to recognizing polylines and curves.
Thoughts
A motivation of this work that was discussed, providing an easier means of integrating sketch recognition into user interfaces, is very similar to that of the quill system that we read about in a previous reading assignment. Other topics that were mentioned in this paper that we have read about in previous assignments included the Sketchpad, Rubine, and Long work that were all cited as previous work that influenced the recognizer discussed within this paper. The previous reading assignment regarded a hybrid recognizer that was mentioned as a future goal within this paper.
I found the fact that a gesture is run through multiple recognizers, one for each primitive shape that the system is capable of recognizing, to be very interesting. Since the results of each are ranked, this would be useful for when a particular gesture is similar to multiple types of shapes, since each shape's likelihood of recognition would then be ranked. Also, the idea of building up complex shapes from a series of primitives seems like a very useful process for recognizing complex gestures.
Wednesday, February 13, 2013
Reading Assignment: What!?! No Rubine Features?: Using Geometric-Based Features to Produce Normalized Confidence Values for Sketch Recognition
Reference Information
Title: "What!?! No Rubine Features?: Using Geometric-Based Features to Produce Normalized Confidence Values for Sketch Recognition
Authors: Brandon Paulson, Panjaj Rajan, Pedro Davalos, Ricardo Gutierrez-Osuna, Tracy Hammond
Citation: "What!?! No Rubine Features?: Using Geometric-Based Features to Produce Normalized Confidence Values for Sketch Recognition", Brandon Paulson, Pankaj Rajan, Pedro Davalos, Ricardo Gutierrez-Osuna, Tracy Hammond.
Summary
This paper discussed a hybrid approach for sketch recognition that combines gesture-based recognition methods and geometric-based recognition methods. Gesture-based recognition uses the stroke properties of the gesture to classify gestures into a particular gesture class. Geometric-based recognition uses the geometric properties of the sketch itself to classify it as a geometric shape. The idea of a hybrid approach was to use the best aspects of each type of recognition to create an improved recognizer for natural sketches with normalized confidence values.
A set of 44 features were used, with 13 gesture-based features (Rubine's) and 31 geometric-based features. Feature subset selection was performed with this set of features in order to determine those features that were the most important for accurate recognition. It was determined that the geometric-based features were selected as being more signification for the given data set than the gesture-based features.
Thoughts
We haven't discussed geometric-based recognition much yet in class, so this paper provided a great, general explanation of what it is. The idea to combine the two sketch recognition methods, gesture-based and geometric-based, into a hybrid recognition system seems like it could be very advantageous due to the combination of the different kinds of techniques. I found it particularly interesting that the feature selection resulted in demonstrating that the geometric features were much more significant than most of the gesture features, even though the Rubine features that were used as the gesture features are a common method for sketch recognition. It would be interesting to determine exactly why the geometric features were chosen as being more significant and if it may be based on the particular data that was used to test the features.
I also liked that this paper built on work that we have seen in previous reading assignments, such as the papers describing the Rubine and Long features. It provided a means for showing ways that the topics discussed in the previous papers have influenced future research.
Title: "What!?! No Rubine Features?: Using Geometric-Based Features to Produce Normalized Confidence Values for Sketch Recognition
Authors: Brandon Paulson, Panjaj Rajan, Pedro Davalos, Ricardo Gutierrez-Osuna, Tracy Hammond
Citation: "What!?! No Rubine Features?: Using Geometric-Based Features to Produce Normalized Confidence Values for Sketch Recognition", Brandon Paulson, Pankaj Rajan, Pedro Davalos, Ricardo Gutierrez-Osuna, Tracy Hammond.
Summary
This paper discussed a hybrid approach for sketch recognition that combines gesture-based recognition methods and geometric-based recognition methods. Gesture-based recognition uses the stroke properties of the gesture to classify gestures into a particular gesture class. Geometric-based recognition uses the geometric properties of the sketch itself to classify it as a geometric shape. The idea of a hybrid approach was to use the best aspects of each type of recognition to create an improved recognizer for natural sketches with normalized confidence values.
A set of 44 features were used, with 13 gesture-based features (Rubine's) and 31 geometric-based features. Feature subset selection was performed with this set of features in order to determine those features that were the most important for accurate recognition. It was determined that the geometric-based features were selected as being more signification for the given data set than the gesture-based features.
Thoughts
We haven't discussed geometric-based recognition much yet in class, so this paper provided a great, general explanation of what it is. The idea to combine the two sketch recognition methods, gesture-based and geometric-based, into a hybrid recognition system seems like it could be very advantageous due to the combination of the different kinds of techniques. I found it particularly interesting that the feature selection resulted in demonstrating that the geometric features were much more significant than most of the gesture features, even though the Rubine features that were used as the gesture features are a common method for sketch recognition. It would be interesting to determine exactly why the geometric features were chosen as being more significant and if it may be based on the particular data that was used to test the features.
I also liked that this paper built on work that we have seen in previous reading assignments, such as the papers describing the Rubine and Long features. It provided a means for showing ways that the topics discussed in the previous papers have influenced future research.
Monday, February 11, 2013
Reading Assignment: Visual Similarity of Pen Gestures
Reference Information
Title: Visual Similarity of Pen Gestures
Authors: A. Chris Long, Jr., James A. Landay, Lawrence A. Rowe, and Joseph Michiels
Citation: "Visual Similarity of Pen Gestures", A. Chris Long, Jr., James A. Landay, Lawrence A. Rowe, Joseph Michiels, Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 360-367, 2000.
Summary
This paper discussed a set of experiments that were conducted in order to create a model for predicting the perceived similarity of gestures. The results of the experiments were used to create the gesture design tool, quill, that was discussed in the previous reading assignment. The motivation behind this research and the tool is that gestures are often difficult for users to remember and recognize. Therefore, the authors wanted to help gesture designers to create improved gestures such that they are easier to recognize by both humans and machines by developing an algorithm to compute the similarity between gestures.
Two experiments with participants were conducted, each designed to determine what properties of a gesture can lead a user to find it similar to other gestures. The experiments were designed with prior work in mind, including work with gesture features (such as Rubine and MDS) and psychological research regarding. The first experiment consisted of showing participants sets of animated gestures and having the participant select the gesture in the set with the least similarity to the others. From the resulting data, a set of features designed to accurately measure similarity was created and a set of equations for prediction were developed. In addition, it was determined that the similarity decisions were participant-dependent. The second experiment was similar to the first, but it allowed the prediction equations from the first experiment to be tested. It was determined that the predictions worked reasonably well and that the perceived similarity can be reasonably related to the features that were calculated for each gesture.
Thoughts
It was very helpful to read the details of the experiments that were briefly mentioned in the previous reading assignment. It made the previous paper much easier to understand, and the amount of detail that was discussed regarding these experiments was very welcoming compared to the lack of details in the previous paper. I found it very interesting that not only was prior work in gesture recognition used to design the experiments, but that psychological research was considered, as well. The development of features based on experiment data seemed like a great idea, as did the fact that a second experiment tested the developments that resulted from the first experiment. It seemed like a very thorough development process.
Some of the details regarding the experiments were debatable, however. For instance, the usage of only a student population for participants, while convenient, may not be the best representation of users for a gesture design tool. In addition, the fact that the gestures were not drawn by the users, but that animations were viewed instead, may have skewed the results, as well. It would be interesting to conduct similar experiments that take these factors into account in order to see whether the results are affected.
Title: Visual Similarity of Pen Gestures
Authors: A. Chris Long, Jr., James A. Landay, Lawrence A. Rowe, and Joseph Michiels
Citation: "Visual Similarity of Pen Gestures", A. Chris Long, Jr., James A. Landay, Lawrence A. Rowe, Joseph Michiels, Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 360-367, 2000.
Summary
This paper discussed a set of experiments that were conducted in order to create a model for predicting the perceived similarity of gestures. The results of the experiments were used to create the gesture design tool, quill, that was discussed in the previous reading assignment. The motivation behind this research and the tool is that gestures are often difficult for users to remember and recognize. Therefore, the authors wanted to help gesture designers to create improved gestures such that they are easier to recognize by both humans and machines by developing an algorithm to compute the similarity between gestures.
Two experiments with participants were conducted, each designed to determine what properties of a gesture can lead a user to find it similar to other gestures. The experiments were designed with prior work in mind, including work with gesture features (such as Rubine and MDS) and psychological research regarding. The first experiment consisted of showing participants sets of animated gestures and having the participant select the gesture in the set with the least similarity to the others. From the resulting data, a set of features designed to accurately measure similarity was created and a set of equations for prediction were developed. In addition, it was determined that the similarity decisions were participant-dependent. The second experiment was similar to the first, but it allowed the prediction equations from the first experiment to be tested. It was determined that the predictions worked reasonably well and that the perceived similarity can be reasonably related to the features that were calculated for each gesture.
Thoughts
It was very helpful to read the details of the experiments that were briefly mentioned in the previous reading assignment. It made the previous paper much easier to understand, and the amount of detail that was discussed regarding these experiments was very welcoming compared to the lack of details in the previous paper. I found it very interesting that not only was prior work in gesture recognition used to design the experiments, but that psychological research was considered, as well. The development of features based on experiment data seemed like a great idea, as did the fact that a second experiment tested the developments that resulted from the first experiment. It seemed like a very thorough development process.
Some of the details regarding the experiments were debatable, however. For instance, the usage of only a student population for participants, while convenient, may not be the best representation of users for a gesture design tool. In addition, the fact that the gestures were not drawn by the users, but that animations were viewed instead, may have skewed the results, as well. It would be interesting to conduct similar experiments that take these factors into account in order to see whether the results are affected.
Wednesday, February 6, 2013
Reading Assignment: "Those Look Similar!" Issues in Automating Gesture Design Advice
Reference Information
Title: "Those Look Similar!" Issues in Automating Gesture Design Advice
Authors: A. Chris Long, Jr., James A. Landay, Lawrence A. Rowe
Citation: "'Those Look Similar!' Issues in Automating Gesture Design Advice", A. Chris Long, Jr., James A. Landay, Lawrence A. Rowe, Proceedings of the 2001 workshop on Perceptive User Interfaces, pp. 1-5, 2001.
Summary
This paper discussed quill, a gesture design tool to help interface designers with the creation of pen-based gestures. It provides the user with unsolicited advice by actively offering users design advice as they create gestures. The advice consists of warnings that appear while the user is creating gestures. It warns if gesture classes are too similar and if a gesture can be easily misrecognized. The idea is to provide a tool that helps novice interface designers to create improved gestures that can be easily recognized by both computers and people.
Various experiments were conducted. The first set of experiments consisted of participants judging similarity between gestures. It allowed a model to be created to recognize gestures that people can easily confuse with other gestures due to similarities. It was determined that the similarity predictions could be wrong, however.
A set of issues regarding the advice was presented, as well. It regarded interface challenges, such as the timing of presenting warnings to users, the amount of advice presented to users, and the content of such advice. Background processes and hierarchical structures were also discussed. It was hoped that the advice presented regarding these issues could be used in the future to improve other gesture techniques.
Thoughts
I think that it's a great idea to create a tool for those unfamiliar with gestures to easily create and improve upon them. I liked that preliminary experiments regarding gesture design were conducted in order to determine a foundation on which to base the tool that was created. In addition, the advice that was presented from the research and experiments that were conducted seems like it could be very helpful to apply to further studies of this nature. However, the paper seemed to be lacking in implementation details about the tool. Also, it was mentioned that a formal evaluation of quill occurred and that some conclusions were made based on it; however, the evaluation itself was never discussed. In addition, some of the conclusions drawn about presenting advice did not seem to explain the reasoning that backed up the conclusion. It would have been helpful to provide a description of how this conclusion was reached, or to have conducted further experiments to test the validity of the conclusion.
Title: "Those Look Similar!" Issues in Automating Gesture Design Advice
Authors: A. Chris Long, Jr., James A. Landay, Lawrence A. Rowe
Citation: "'Those Look Similar!' Issues in Automating Gesture Design Advice", A. Chris Long, Jr., James A. Landay, Lawrence A. Rowe, Proceedings of the 2001 workshop on Perceptive User Interfaces, pp. 1-5, 2001.
Summary
This paper discussed quill, a gesture design tool to help interface designers with the creation of pen-based gestures. It provides the user with unsolicited advice by actively offering users design advice as they create gestures. The advice consists of warnings that appear while the user is creating gestures. It warns if gesture classes are too similar and if a gesture can be easily misrecognized. The idea is to provide a tool that helps novice interface designers to create improved gestures that can be easily recognized by both computers and people.
Various experiments were conducted. The first set of experiments consisted of participants judging similarity between gestures. It allowed a model to be created to recognize gestures that people can easily confuse with other gestures due to similarities. It was determined that the similarity predictions could be wrong, however.
A set of issues regarding the advice was presented, as well. It regarded interface challenges, such as the timing of presenting warnings to users, the amount of advice presented to users, and the content of such advice. Background processes and hierarchical structures were also discussed. It was hoped that the advice presented regarding these issues could be used in the future to improve other gesture techniques.
Thoughts
I think that it's a great idea to create a tool for those unfamiliar with gestures to easily create and improve upon them. I liked that preliminary experiments regarding gesture design were conducted in order to determine a foundation on which to base the tool that was created. In addition, the advice that was presented from the research and experiments that were conducted seems like it could be very helpful to apply to further studies of this nature. However, the paper seemed to be lacking in implementation details about the tool. Also, it was mentioned that a formal evaluation of quill occurred and that some conclusions were made based on it; however, the evaluation itself was never discussed. In addition, some of the conclusions drawn about presenting advice did not seem to explain the reasoning that backed up the conclusion. It would have been helpful to provide a description of how this conclusion was reached, or to have conducted further experiments to test the validity of the conclusion.
Subscribe to:
Comments (Atom)