Kirchhoff's Pen: a pen-based circuit analysis tutor
SBIM '07 Proceedings of the 4th Eurographics workshop on Sketch-based interfaces and modeling
Newton's Pen: A pen-based tutoring system for statics
Computers and Graphics
Formalizing Recognition of Sketching Styles in Human Centered Systems
KES '07 Knowledge-Based Intelligent Information and Engineering Systems and the XVII Italian Workshop on Neural Networks on Proceedings of the 11th International Conference
Intelligent understanding of handwritten geometry theorem proving
Proceedings of the 15th international conference on Intelligent user interfaces
Parsing ink annotations on heterogeneous documents
SBM'06 Proceedings of the Third Eurographics conference on Sketch-Based Interfaces and Modeling
| Hi-index | 0.00 |
Pen-based computer interaction is becoming increasingly ubiquitous as evidenced by the growing interest in Tablet PC's, electronic whiteboards and PDA's. Many of these devices now come equipped with robust hand-writing recognizers. However, a problem that remains largely unsolved is the recognition of graphical input such as schematic sketches and diagrams. When faced with such input, these devices either leave the pen strokes uninterpreted, or offer only limited support for recognition, while placing many unnatural constraints on the way the user draws. These constraints might include limitations to single-stroke objects, or the need for user involvement in separating different visual objects. In this work, we present a new approach for recognizing hand-drawn, diagrammatic sketches. The key advance is an integrated sketch parsing and recognition model designed to enable natural and fluid pen-based computer interaction. With this approach, the stream of pen strokes is first examined to identify certain delimiter patterns called “markers.” These then anchor a spatial analysis which groups the remaining strokes into distinct clusters, each representing a single visual object. Finally, a shape recognizer is used to find the best interpretations of the clusters. This approach eliminates many of the unnatural constraints imposed by existing sketch understanding systems. To demonstrate our techniques, we have built SimuSketch, a sketch-based interface for Matlab's Simulink package, and VibroSketch, a sketch-based interface for analyzing vibratory mechanical systems. In both systems, users can construct functional engineering models by simply sketching them on a computer screen. Users can then interactively manipulate their sketches to change model parameters and run simulations. Our user studies have indicated that even novice users can effectively utilize these systems to solve real engineering problems, without having to know much about the underlying recognition techniques.