We use a template-based detection system to capture the fine details that distinguish gestures. To model the visual variation within a gesture type, such as pointing, we assemble a representative set of templates. Fig. 4 illustrates the possible variations among pointing gestures. Our detector relies on the Laplacian of Gaussian to create high contrast responses along contours (bottom row of Fig. 4). Aligning a LoG template with a matched gesture results in a strong correlation.

​

To learn the representative set of templates for a gesture, we compute the PCA decomposition of a set of training examples. Fig. 5 shows several principle components of the pointing gesture displaying significant variations. We identify the representative set of templates as those with the largest projection on the principle components. The gestures in Fig. 4 represent the characteristic subset of pointing gestures out of approximately 100 instances.

We use an efficient version of normalized cross-correlation with the templates to vote for detections over scale and rotations. Detection hypotheses with a high score are verified with a support vector machine trained over HoG features.

​

The detection results in Tab. 1 closely reflect those in the ground-truth. The high accuracy in the Precision-Recall curves of Fig. 6 increases for larger sets of learned templates. Example detections for each of the four gesture types are shown in ,b>Fig. 7 The last column of each row in Fig. 7 shows false detections. Notice how the pattern is visually similar to the artist's rendering of the gesture.

​

From here we will evaluate the detection system on other copies of the Sachsenspiegel to provide a systematic comparison of gesture usage. We plan to follow this with high-level scene analysis to automatically determine if gestures are used in a specific legal context.