Reading Assignment: Sketch Recognition Algorithms for Comparing Complex and Unpredictable Shapes

Reference Information
Title: "Sketch Recognition Algorithms for Comparing Complex and Unpredictable Shapes"
Authors: Martin Field, Stephanie Valentine, Julie Linsey, Tracy Hammond
Citation: "Sketch Recognition Algorithms for Comparing Complex and Unpredictable Shapes", Martin Field, Stephanie Valentine, Julie Linsey, Tracy Hammond, Proceedings of the 22nd International Joint Conference on Artificial Intelligence, pp. 2436-2441, 2011

Summary
This paper discussed applying sketch recognition techniques to the field of engineering education. Specifically, a system called Mechanix was created to recognize freehand sketches of engineering problems for educational purposes, such as for completing homework and exams. Feedback is provided to students on their sketches based on comparing the sketches to a solution provided by the class grader/professor by also using Mechanix. One of the benefits of Mechanix is that it provides an environment for sketching engineering problems that is very similar to the traditional pen and paper approach, providing a natural means of completing assignments that is easier to grade and contains instant feedback.

The algorithms used for Mechanix are designed to identify bodies (in free-body diagrams) and trusses, as well as to use recognition to compare them to the solution sketches provided by the grader. The system uses an online recognition process that runs after each separate stroke is sketched. Identification of bodies occurs by using the pre-existing PaleoSketch system to find any strokes that form a closed shape, which constitutes a body. Then, the body is compared for similarity to the single template solution provided by the grader by using a combination of the Hausdorff distance, the modified Hausdorff distance, and the Tanimoto coefficient. Trusses are recognized as a collection of polygons that share sides, and are identified by using PaleoSketch to build a connection graph to find at least two polygons that share an edge. The identified truss is then compared to the single template provided by the grader, using the connection graph and the properties of the sketch.

Mechanix was evaluated via user testing and with use during a single section of the ENGR 111 class. The students that used the system had positive feedback and it was determined that the recognition is relatively accurate with little possibility of being able to trick the system.

Thoughts
Mechanix is an application that has very practical purposes of improving both the educational experience for engineering students and the grading process for engineering graders. It was very refreshing to read about such a useful application that I have some prior experience with, and that the sketch recognition techniques that we've been learning can be applied to. Also, it was very useful that the prior work section of this paper conveniently mentioned how Mechanix improves upon or uses each of the prior systems, instead of simply discussing those systems.

In addition, the fact that part of the evaluation was able to occur by having the system used within an actual engineering class was very interesting and potentially quite beneficial. Since Mechanix was designed for that purpose, being able to test its usage within the same domain that it is intended to be used in, by potential actual users of the system, is a great opportunity to receive more accurate evaluation feedback.