Hand Tracking Lab

In the Quest: Put your controllers aside and make sure hand tracking is enabled. Hold both bare hands where the headset cameras can see them. Move and turn them slowly, then try covering a fingertip.

Lab 003: The purple line is the Quest's actual hand pointing ray (targetRaySpace), not the direction of your index finger. The sign shows its left/right heading and up/down elevation in the room's local coordinates, and its movement in degrees. With your fingers open (at least 70 mm apart), the lab keeps updating the original pointing direction. It holds that starting direction once your fingertips are within 50 mm. “At select” records how far the beam had moved when Quest recognized the pinch; “Peak before” remembers the largest change up to that moment. Values remain visible until a later recognized pinch. The gold line still measures thumb-to-index separation; green means the lab's 20 mm threshold, not Quest recognition. The pale finger-line shows the original index-finger direction.

Move and twist the readout: It starts to your left and stays in the room instead of following your face. Aim a hand's Quest pointing beam at the panel, pinch thumb and index finger to grab, then move your hand to reposition it and rotate your wrist to turn or tilt the sign. Release to leave it in place and at that angle. You can also pinch while touching the panel directly. It resets to its starting position and orientation each time you enter passthrough. Manipulating the panel does not change pinch calculations, although the grab pinch is counted.

Lab 004 · Pick up objects: Three objects appear in front of you. Reach your thumb and index finger near one, or aim the purple Quest beam at it. Use a Quest-recognized select pinch to pick it up. Move your hand to move it; wrist rotation turns it. Release the selection to leave it floating in its new position. The sign is still movable and rotatable, and the complete 25-joint hand skeleton remains visible. The Quest supplies joints, pointer poses and selection events—not a built-in object-pickup result. Our lab detects which object was near your fingers or ray. The readout labels these sources separately. A squeeze event or gripSpace may not be available for a hand; the sign reports what the browser actually exposes. No physical-world collisions or gravity are simulated.

Lab 009 · Native thumb microgestures: In Quest Browser, a hand gamepad may provide five very brief button pulses: buttons 5/6 for left/right swipes, 7/8 for forward/backward swipes, and 9 for thumb tap. We poll these every WebXR frame and count each rising edge. Button 0 (select) is shown separately; button 4 (menu) is reserved for the system and never intercepted. If the browser exposes fewer than 10 hand gamepad buttons, the sign reports missing inputs rather than inventing recognitions. The 25-joint skeleton, hand models, rays and movable diagnostic sign remain available, but no gestures or recordings are uploaded. The native Interaction SDK offers more features than standard WebXR; this lab tests only values available through the browser.

Lab 008 · Recognizable hand poses: WebXR provides 25 joint poses and separate select/squeeze events. Our software estimates curl from finger-chain geometry and uses configurable thresholds to identify six candidate shapes: Open palm, Fist, Index point, Peace/V, Thumbs-up, and OK circle. When a candidate is held steadily for 150–600 ms, it becomes confirmed. Unclassified and missing-tracking states are explicit. The 008.3 thumbs-up rule requires all four non-thumb fingers curled, thumb-tip clearance greater than 100% of tracked palm width, and thumb distance from the curled-finger center greater than 200% of tracked palm width; thumb extension remains a diagnostic value only; the lab never claims to receive a Quest-provided pose label. The full skeleton, published hand models, Quest target rays, and movable/rotatable sign remain available. Earlier experiments and Dinoscores are not modified.

Lab 007 · Continuous pinch: The Quest supplies raw thumb/index joint poses and native selectstart/selectend events. We calculate the gap in millimeters and turn it into a 0–100% estimated pinch strength using the settings above. The floating, draggable sign shows both hands, approximate non-thumb finger curl, and a rolling 6–20 second graph. White select-start/end markers are actual browser events. The event's gap is captured from its XRFrame when available, otherwise the most recent tracking frame is used. No native pinch-pressure or continuous strength signal is asserted. The overlay hand models, 25 raw joint dots, and Quest aim rays remain available. The 006.1 calibration does not modify these measurements.

Lab 005.3 · Finger contact fit: As in 005.2, reach with an open hand, curl several fingers to grab, and reopen to release. The hand mask keeps real fingers visible in front of the held object. New: the lab uses finger-tip positions to make small, bounded adjustments to the held object so its visible surface better meets the curled fingers. Choose Original 005.2 placement for a direct comparison, or Balanced / Stronger fit. Because you observed that the Quest's fingertip dots stop short of your actual fingertips, there is an optional 0 / 8 / 16 mm fingertip allowance. This is an approximation, not a way to reshape the real passthrough video or change the physical pose of your hand. Your 25 tracked points, connecting lines, depth mask, and movable sign remain. The sign will show the approximate contact mismatch in millimeters after a grab. Dinoscores is unchanged.

Lab 006 · Hand appearance: It uses Meta's publicly released generic-hand model through the standard Three.js WebXR hand-model factory. This is not the Meta native SDK hand renderer. Select separate hand-model, skeleton, and Quest-ray layers before entering. A simple joint-sphere representation is available if the model asset has not loaded. The sign shows which input fields are actually reported by WebXR, and can be moved and twisted by aiming and pinching. This is the only experiment that lazily uses external model and rendering libraries; Labs 001–005 remain independent.

Lab 006.1 · Hand size calibration: The calibrated skeleton starts larger than Quest's raw skeleton so the difference is visible. Cyan points and connecting lines show WebXR's original positions; gold marks show the lab's adjusted display positions. The wrist remains fixed; overall hand size, finger lengths, and additional fingertip reach can be adjusted independently for left and right. You can also set all three values to their neutral positions (100%, 100%, 0 mm). The skinned Meta model attempts to follow the corrected display joints, but the published mesh cannot perfectly reproduce your actual hand surface. Leave Lab 006.0 selected to compare with its untouched appearance. No calibration changes underlying Quest tracking or gestures.

Change experiments: use the Quest's built-in left-palm menu gesture to exit immersive mode. Look at your left palm, pinch thumb and index finger until the menu icon fills, then release. The experiment menu returns here. This reserved system gesture is not part of our lab.