A sketch recognition interface that recognizes hundreds of shapes in course-of-action diagrams

  • Authors:

  • Tracy Hammond;Drew Logsdon;Joshua Peschel;Joshua Johnston;Paul Taele;Aaron Wolin;Brandon Paulson

  • Affiliations:

  • Texas A&M University, College Station, TX, USA;Texas A&M University, College Station, TX, USA;Texas A&M University, College Station, TX, USA;Texas A&M University, College Station, TX, USA;Texas A&M University, College Station, TX, USA;Texas A&M University, College Station, TX, USA;Texas A&M University, College Station, TX, USA

  • Venue:
  • CHI '10 Extended Abstracts on Human Factors in Computing Systems
  • Year:
  • 2010

Quantified Score

Hi-index 0.00

Visualization

Abstract

Sketch recognition is the automated recognition of hand drawn diagrams. Military course-of-action (COA) diagrams are used to depict battle scenarios. The domain of military course of action diagrams is particularly interesting because it includes tens of thousands of different geometric shapes, complete with many additional textual and designator modifiers. Existing sketch recognition systems recognize on the order of at most 20 different shapes. Our sketch recognition interface recognizes 485 different freely drawn military course-of-action diagram symbols in real time, with each shape containing its own elaborate set of text labels and other variations. We are able to do this by combining multiple recognition techniques in a single system. When the variations (not allowable by other systems) are factored in, our system is several orders of magnitude larger than the next biggest system. On 5,900 hand-drawn symbols drawn by 8 researchers, the system achieves an accuracy of 90% when considering the top 3 interpretations and requiring every aspect of the shape (variations, text, symbol, location, orientation) to be correct.