Static Pose Detection

Overview

This page describes an approach to detecting hand poses.

Finger Position Terminology

The following are standard finger posing terms and how to detect them.

Finger Flexion

Finger Flexion is bending of a finger in towards a fist. As such it can be measured by detecting the angle from the metacarpal to the proximal joint as measured in the direction of closing the hand into a fist.

_{Note: an alternative measurement would be the angle of the proximal to the plane defined by the palm. This may be superior in measuring the flexion of the thumb, as thumb flexion also moves the thumbs metacarpal.}

Finger Flexion of 0 degrees

Finger Flexion of 90 degrees

Finger Curl

Finger Curl is the bending of the finger at the intermediate and distal joints. As such it can be measured by detecting the angle from the proximal to the distal joint as measured in the direction of closing the hand into a fist.

Finger Curl of 0 degrees

Finger Curl of 180 degrees

Finger Abduction

Finger Abduction is the separation of fingers at the proximal joint. As such it can be measured by detecting the angle between the two selected fingers at the proximal joint as measured orthogonal to the direction of closing the hand into a fist.

Index to Middle Abduction of 0 degrees

Index to Middle Abduction of 30 degrees

Theory of Operation

Pose detection would occur by taking the current tracked hand pose (in the XRHandTracker) and comparing it to a set of hand-poses. The pose with the smallest error is checked against a tolerance-threshold, and if it's a good enough match, the pose is reported.

Hand Pose Rules

The hand pose would consist of a set of finger rules as follows:

• Finger Flexion Angle:
  • Finger: [Thumb, Index, Middle, Ring, Pinky]
  • Angle: [0..90] degrees
  • Tolerance: [0..90] degrees
• Finger Curl Angle:
  • Finger: [Thumb, Index, Middle, Ring, Pinky]
  • Angle: [0..180] degrees
  • Tolerance: [0..180] degrees
• Finger Abduction Angle:
  • Finger1: [Thumb, Index, Middle, Ring, Pinky]
  • Finger2: [Thumb, Index, Middle, Ring, Pinky]
  • Angle: [0..90] degrees
  • Tolerance: [0..90] degrees
• Finger Tip Distance:
  • Finger1: [Thumb, Index, Middle, Ring, Pinky]
  • Finger2: [Thumb, Index, Middle, Ring, Pinky]
  • Distance: [0..200] mm
  • Tolerance: [0..200] mm

Error Calculation

Each rule in the hand pose returns an error, and the hand-pose error is the sum of the errors of each rule.

The rule error value is calculated by comparing the measured value (angle or distance) against the expected value and producing an error using the following function:

float to_error(float measured, float expected, float tolerance)
{
    // Reject invalid tolerances
    if (tolerance <= 0)
        return 1;

    // Calculate the distance
    float d = std::fabs(measured - expected) / tolerance;

    // Clamp to the range 0..1
    d = std::clamp(d, 0, 1);

    // Calculate smoothstep (cubic hermite) response
    return d * d * (3 - 2 * d);
}

This produces a smooth from 0 to 1 as the error rises to the tolerance.

Example Hand Poses

Pointing

Description	Type	Parameters
Thumb Pointing Straight Out	Finger Flexion Angle	Finger=Thumb, Angle=0, Tolerance=30
Thumb Held Straight	Finger Curl Angle	Finger=Thumb, Angle=0, Tolerance=20
Thumb Pointing Up	Finger Abduction Angle	Finger1=Thumb, Finger2=Index, Angle=80, Tolerance=30
Index Pointing Straight Out	Finger Flexion Angle	Finger=Index, Angle=0, Tolerance=20
Index Held Straight	Finger Curl Angle	Finger=Index, Angle=0, Tolerance=20
Middle Flexed In	Finger Flexion Angle	Finger=Middle, Angle=70, Tolerance=50
Ring Flexed In	Finger Flexion Angle	Finger=Ring, Angle=70, Tolerance=50
Little Flexed In	Finger Flexion Angle	Finger=Little, Angle=70, Tolerance=50

Spock

Description	Type	Parameters
Thumb Pointing Straight Out	Finger Flexion Angle	Finger=Thumb, Angle=0, Tolerance=30
Thumb Held Straight	Finger Curl Angle	Finger=Thumb, Angle=0, Tolerance=20
Index Pointing Straight Out	Finger Flexion Angle	Finger=Index, Angle=0, Tolerance=20
Index Held Straight	Finger Curl Angle	Finger=Index, Angle=0, Tolerance=20
Middle Pointing Straight Out	Finger Flexion Angle	Finger=Middle, Angle=0, Tolerance=20
Middle Held Straight	Finger Curl Angle	Finger=Middle, Angle=0, Tolerance=20
Ring Pointing Straight Out	Finger Flexion Angle	Finger=Ring, Angle=0, Tolerance=20
Ring Held Straight	Finger Curl Angle	Finger=Ring, Angle=0, Tolerance=20
Little Pointing Straight Out	Finger Flexion Angle	Finger=Little, Angle=0, Tolerance=20
Little Held Straight	Finger Curl Angle	Finger=Little, Angle=0, Tolerance=20
Thumb Separated from Index	Finger Abduction Angle	Finger1=Thumb, Finger2=Index, Angle=80, Tolerance=30
Index Next to Middle	Finger Abduction Angle	Finger1=Index, Finger2=Middle, Angle=0, Tolerance=20
Middle Separated from Ring	Finger Abduction Angle	Finger1=Middle, Finger2=Ring, Angle=25, Tolerance=20
Ring Next to Little	Finger Abduction Angle	Finger1=Ring, Finger2=Little, Angle=0, Tolerance=20

Pinch

Description	Type	Parameters
Thumb Close to Index	Finger Tip Distancee	Finger1=Thumb, Finger2=Index, Distance=10, Tolerance=10