"Fluid Sketches: Continuous Recognition and Morphing of Simple HandDrawn Shapes"

by James Arvo, Kevin Novins

Summary

This paper presents an approach to tightly couple gesture recognition with rapid morphing to provide a new and subjectively different form of feedback. The feedback is provided as each pen stroke is drawn, so that immediate adjustments can be made if the result is not what was intended.

A family of ordinary different eqations (ODEs) is used to determine how a user-drawn shape changes with time as a result of continuous recognition and morphing. The parametric ODE is referred to as the fluid sketching equation. Points are continuous moved towards its corresponding ideal position that reflects the curve the user is attempting to draw. qs(t) is the new position of ps at time t. It is this feedback of q influencing its own subsequent time evolution that leads to modeling q as a differential equation.

Recognition stage attempts to classify the user-drawn stroke into a fragment of a known shape (circle, box, line segment). Best fit of the stroke to each shape is calculated. The metric is the sum of the squares of the distances from each sampled point on the curve to the closest-matching shape. Least-squares fit for circle is found by applying a linear equation, while for box, relaxation is used (each point exerts a "spring" force to the nearest face or vertex and use gradient descent to get to a configuration in which the total net force is nearly zero - most balanced state).

The matching is based on the entire stroke, but the recent portion is weighed more heavily. The simplest strategy for f is to simply move in the direction of the closest point on the ideal shape. Viscosity v is to describe how fast the point on the stroke moves toward the ideal shape S. v=0 means instantaneous snapping to the ideal shape, while v=infinity means that the user-drawn strokes are retained
unmorphed.

In situations where accuracy of placement is less important, the majority of users in this study preferred fluid sketching.


Discussion

The approach introduced in the paper enables the user to see the cleaned shape portion without even finishing the stroke. However, this may distract user's attention and also distort the stroke at an early stage that may yield a misrecognition.