Title: Markerless Augmented Reality Registration for Surgical Guidance: A Multi-Anatomy Clinical Accuracy Study

Abstract

Purpose: This study presents the development and clinical assessment of a depth-based, markerless augmented reality (AR) registration system deployed on a head-mounted display (HMD). The primary objective is to evaluate the system’s precision in real-world surgical environments, specifically focusing on anatomical structures with small surface areas or low curvature.

Methods: Utilizing the HoloLens 2, the proposed pipeline aligns Articulated HAnd Tracking (AHAT) depth data with Computed Tomography (CT)-generated skin meshes. This alignment process comprises three key steps: correction of depth bias, a concise human-in-the-loop initialization phase, and both global and local registration procedures. To validate the surface-tracing error metric, we compared relative "skin-to-bone" distances measured by an AR-tracked tool against CT ground truth data using leg and foot models. Subsequently, we conducted seven intraoperative target trials during the initial phases of fibula free-flap harvesting and mandibular reconstruction surgeries. These trials involved two procedures on feet, three on ears, and two on legs, with more than 500 data points collected per trial.

Results: Preclinical validation demonstrated strong concordance between distances traced via AR and those derived from CT scans. For the leg, the median absolute difference (|Delta d|) was 0.78 mm with a root mean square error (RMSE) of 0.97 mm; for the feet, these figures were 0.80 mm and 1.20 mm, respectively. In clinical settings, the median per-point error was recorded at 3.9 mm. When analyzed by anatomical site, median errors were 3.2 mm for the feet, 4.3 mm for the ear, and 5.3 mm for the lower leg. The coverage rates for errors within 5 mm ranged from 92-95% for the feet, 84-90% for the ear, and 72-86% for the lower leg. A statistically significant difference was observed between the feet and the lower leg, with a median difference of approximately 1.1 mm (p < 0.001).

Conclusion: In live surgical contexts, the depth-only, markerless AR pipeline achieved a median error of approximately 3–4 mm across the feet, ear, and lower leg without the need for fiducial markers. These results approach the standard clinical error thresholds associated with moderate-risk procedures. By combining human-guided initialization with a global-to-local registration strategy, the system facilitates accurate alignment on targets characterized by small size or low curvature, thereby enhancing the clinical viability of markerless AR guidance.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC