Augmented Reality in the Operating Room: How Surgeons See Inside the Body

Medical note: this article and its images were made by AI. People can correct them, but they remain automated and have not been reviewed by a doctor, so treat them as general information, not medical advice. For a diagnosis, treatment or procedure, see a doctor.

In the traditional operating room, a surgeon’s level of precision is often limited by “mental mapping”—the process of looking at a 2D CT scan or MRI on a wall-mounted monitor and then trying to visualize where those internal structures lie beneath the patient’s skin. This cognitive load can lead to minute errors or longer operative times.

Augmented Reality (AR) is transforming this workflow by overlaying digital anatomical data directly onto the surgical field. Unlike Virtual Reality (VR), which creates a totally simulated environment, AR allows surgeons to maintain their natural line of sight while seeing “through” tissue to locate tumors, blood vessels, and bone structures in real-time.

Your reading tools

  • Simplify
    Any section in plain language
  • Summarize
    The key points at a glance
  • FAQ
    Quick answers to common questions
  • New
    i
    Spotted a mistake? We fix it in minutes!

Written with AI from the sources listed at the end.

Table of Contents

  1. How AR Technology Works in Surgery
  2. Applications in Plastic and Reconstructive Surgery
  3. General Surgery and Internal Navigation
  4. Virtual Reality in Surgical Training
  5. Challenges and Limitations
  6. Summary of Key Takeaways
  7. Sources

How AR Technology Works in Surgery

The “magic” of seeing inside the body relies on three core technological components:

  1. Image Fusion: Pre-operative scans (CT, MRI, or PET) are converted into 3D holograms.
  2. Registration: The system aligns the 3D hologram with the patient’s physical body using “spatial anchors” like the umbilicus, bony landmarks, or markers called fiducials [2] [3].
  3. Visualization: Surgeons view these overlays through Head-Mounted Displays (HMDs) like the Microsoft HoloLens 2 or Apple Vision Pro, or via Projected Augmented Reality (PAR), which beams the data directly onto the patient’s skin [1].

Applications in Plastic and Reconstructive Surgery

Plastic surgery requires extreme precision, particularly when harvesting tissue “flaps” to rebuild parts of the body. AR is currently being used to map the “perforators”—the tiny blood vessels that supply blood to moved tissue.

  • DIEP Flap Breast Reconstruction: Surgeons use AR to visualize the exact location of abdominal blood vessels before making an incision. So far this has been tried in small series: a 2025 report used a Meta Quest 3 headset in 3 DIEP flap operations, and its authors present it as support for the hypothesis that AR will shorten flap harvest and improve safety, not as proof [3].

  • Craniofacial Surgery: For procedures like cleft lip repair, a University of California, Irvine team has built a projected AR system that placed surgical markings on anatomical models with sub-millimeter precision, and a machine-learning model that designs cleft lip markings, which they plan to combine; two of the authors co-founded a company developing the technology [1].

General Surgery and Internal Navigation

While plastic surgery often deals with the body surface, general surgeons are using AR to navigate mobile organs.

  • Stoma Creation: In a first series of 14 patients at Weill Cornell, surgeons used a HoloLens 2 headset to overlay a 3D map of the bowel, bones and vessels and mark the path to the target loop on the skin; the right loop was found in every case, without open surgery [2].

  • Laparoscopic Liver Surgery: Systems like “Smartliver” overlay 3D models of the liver and its blood vessels onto the laparoscopic video. In a UCL study with 13 patients, surgeons found the overlay intuitive and potentially helpful for decisions, but it was still aligned by hand; automating the alignment and predicting how the liver deforms are the next steps [5].

  • Minimalist Setup: In the first US series of minimally invasive operations with the Apple Vision Pro (41 cases at UC San Diego), surgeons used up to three virtual displays of the laparoscopic or endoscopic video, placed where they wanted in their field of view, instead of wall monitors; they reported a low workload and there were no complications, though the study had no comparison group [4].

This technological shift goes beyond the TV drama and represents a move toward “GPS for the human body.”

Virtual Reality in Surgical Training

The impact of VR varies across specialties, but the most significant gains are seen in high-precision fields like orthopedics and ophthalmology.

1. Orthopedic Surgery

Orthopedics requires a deep understanding of 3D spatial relationships and haptic precision. A 2025 meta-analysis of 23 randomized trials (1,091 participants) in the Journal of Medical Internet Research found that VR-based education gave higher knowledge, clinical operation and surgical design scores than traditional teaching, though the trials varied widely [6].

2. Ophthalmology

In eye microsurgery the evidence is strongest: a 2025 meta-analysis of 17,623 cataract operations found that posterior capsular rupture (PCR), an important complication, was less frequent after trainees practiced on the Eyesi simulator (odds ratio 0.71) [7].

3. Plastic and Reconstructive Surgery

Plastic surgery often involves intricate “what-if” scenarios. VR allows residents to visualize patient-specific anatomy in 3D. Published evidence for plastic surgery training is thinner than for orthopedics and eye surgery.

Challenges and Limitations

Table: Comparison of AR Display Methods
FeatureHMD (Headsets)Projected AR (PAR)
ImmersionHigh (3D Holograms)Moderate (2D/3D Surface)
ErgonomicsPossible Neck StrainNo Extra Weight
Team AccessIndividual UserVisible to Entire Team

Despite the benefits, AR is not yet standard in every hospital. Key hurdles include:

  • Soft Tissue Deformation: Unlike bones, internal organs move when a patient breathes or when a surgeon touches them. Maintaining the “alignment” of the hologram to a moving organ is a major technical challenge [5].

  • Ergonomics: Head-mounted displays have a restricted field of view, can be uncomfortable to wear and show the overlay to one user only. Projected AR (PAR) instead projects markings directly onto the patient, with no headset and visible to the whole team [1].

Understanding these advanced tools can help demystify the experience for those preparing for a procedure.

Summary of Key Takeaways

  • Real-Time Data Integration: AR overlays 3D patient anatomy (from CT/MRI) directly onto the surgical site, allowing surgeons to “see through” skin and tissue.

  • Eliminating Distraction: Headsets like the Apple Vision Pro let surgeons see the camera video without looking away at wall monitors; in a first series, surgeons reported a low workload [4].

  • Precision in Reconstruction: In plastic surgery, AR is being tested to locate blood vessels and plan incisions; early reports are small and outcome benefits are not yet proven [3].

  • Collaborative Visibility: Projected AR (PAR) allows the entire surgical team to see the guides simultaneously without requiring individual headsets [1].

Action Plan for Patients and Practitioners:

  1. For Patients: If undergoing complex reconstructive or abdominal surgery, ask your surgeon if they utilize “image-guided navigation” or AR for preoperative planning.
  2. For Surgeons: Evaluate the ergonomic trade-offs between Head-Mounted Displays (maximum immersion) and Projected AR (better team collaboration/comfort).
  3. Integration: Ensure preoperative imaging is high-resolution (thin-slice CT/MRI) to maximize the accuracy of the AR hologram registration.

Augmented reality could move surgery from mental mapping toward data-guided navigation, but most of the systems described here are still in small early studies.

Table: Summary of AR Benefits in Modern Surgery
Benefit CategoryImpact on Surgical Outcome
PrecisionReal-time visualization of sub-surface anatomy and blood vessels.
Cognitive LoadReduces the need for manual mental mapping from 2D scans.
EfficiencyExpected to shorten the search for structures like perforators; not yet shown in large studies.
SafetyAims to help avoid critical structures; liver systems are at an early clinical stage.

Sources