"Sketch Recognition User Interfaces: Guidelines for Design and Development"

by Christine Alvarado

Summary

This paper addresses both HCI and sketch recognition by introducing a recognition-based diagram creation tool that robustly recognizes naturally drawn diagrams and automatically imports them into PowerPoint. Recognition is done by a multi-domain sketch engine called SketchREAD. Users create diagrams consisting of shapes (ellipses and quadrilaterals) and connectors (lines and arrows) on Tablet PCs using this tool.

To enable seamless interaction, the diagram creation tool and PPT is automatically synchronized to contain the same information. This tool also provides editing capabilities including move and delete. Pen-based editing commands are designed. Users switch between edit mode and sketch mode by pressing buttons on the winodw. Online edit mode is introduced to allow users sketch while in edit mode. The user holds down the pen in sketch mode to enter online edit mode, then she selects the items, when she lifted the pen, she can draw new items, however, the selected items remain highlighted, indicating that she can move or delete them using the same gestures in edit mode. Both recognized (cleaned) strokes and unrecognized (original) strokes appear on the slides.

Design guidelines: (1) Display recognition results only when the user is done sketching. Whether the diagram is finished is indicated by changing of window focus, or explicited by the user by pressing "show" button. (2) Provide obvious indications to distinguish free sketching from recognition. (3) Restrict recognition to a single domain until automatic domain detection becomes feasible. (4) Incorporate pen-based editing as much as possible (copy, paste, alignment, resizing). (5) Sketching and editing should use distinct pen motions. (6) SkRUIs require large buttons.


Discussion

Not touching the recognition techniques in detail, the paper discusses mainly the UI design and evalution of the sketch recognition tool. The design guidelines are useful.

The online edit mode may still be confusing to the user, it's better to provide a more straightforward way to switch between editing and sketching, like the trigger-action used in LADDER.