Augmented Reality in the Operating Room: How Surgeons See Inside the Body
Medical note: this article and its images were made by AI. People can correct them, but they remain automated and have not been reviewed by a doctor, so treat them as general information, not medical advice. For a diagnosis, treatment or procedure, see a doctor.
In the traditional operating room, a surgeon’s level of precision is often limited by “mental mapping”—the process of looking at a 2D CT scan or MRI on a wall-mounted monitor and then trying to visualize where those internal structures lie beneath the patient’s skin. This cognitive load can lead to minute errors or longer operative times.
Augmented Reality (AR) is transforming this workflow by overlaying digital anatomical data directly onto the surgical field. Unlike Virtual Reality (VR), which creates a totally simulated environment, AR allows surgeons to maintain their natural line of sight while seeing “through” tissue to locate tumors, blood vessels, and bone structures in real-time.
Your reading tools
- SimplifyAny section in plain language
- SummarizeThe key points at a glance
- FAQQuick answers to common questions
- NewSpotted a mistake? We fix it in minutes!
Written with AI from the sources listed at the end.
Table of Contents
- How AR Technology Works in Surgery
- Applications in Plastic and Reconstructive Surgery
- General Surgery and Internal Navigation
- Virtual Reality in Surgical Training
- Challenges and Limitations
- Summary of Key Takeaways
- Sources
How AR Technology Works in Surgery
The “magic” of seeing inside the body relies on three core technological components:
- Image Fusion: Pre-operative scans (CT, MRI, or PET) are converted into 3D holograms.
- Registration: The system aligns the 3D hologram with the patient’s physical body using “spatial anchors” like the umbilicus, bony landmarks, or markers called fiducials [2] [3].
- Visualization: Surgeons view these overlays through Head-Mounted Displays (HMDs) like the Microsoft HoloLens 2 or Apple Vision Pro, or via Projected Augmented Reality (PAR), which beams the data directly onto the patient’s skin [1].
While Virtual Reality (VR) creates a completely simulated digital environment, Augmented Reality (AR) overlays digital information onto the real-world surgical field. This allows surgeons to see internal anatomy, such as tumors or blood vessels, without losing sight of the patient.
This process is called registration. Surgeons use ‘spatial anchors’ like bony landmarks, the navel, or markers called fiducials to align pre-operative CT or MRI scans perfectly with the patient’s physical position.
Applications in Plastic and Reconstructive Surgery
Plastic surgery requires extreme precision, particularly when harvesting tissue “flaps” to rebuild parts of the body. AR is currently being used to map the “perforators”—the tiny blood vessels that supply blood to moved tissue.
-
DIEP Flap Breast Reconstruction: Surgeons use AR to visualize the exact location of abdominal blood vessels before making an incision. So far this has been tried in small series: a 2025 report used a Meta Quest 3 headset in 3 DIEP flap operations, and its authors present it as support for the hypothesis that AR will shorten flap harvest and improve safety, not as proof [3].
-
Craniofacial Surgery: For procedures like cleft lip repair, a University of California, Irvine team has built a projected AR system that placed surgical markings on anatomical models with sub-millimeter precision, and a machine-learning model that designs cleft lip markings, which they plan to combine; two of the authors co-founded a company developing the technology [1].
In DIEP flap procedures, AR lets surgeons see the abdominal blood vessels mapped on preoperative scans before making an incision. Surgeons expect this to shorten the operation and improve safety, but so far it has been tested in only a few operations.
Not yet in routine care. Researchers have projected cleft lip surgical markings onto anatomical models with sub-millimeter precision and are adding an AI model that designs the markings; results in patients have not been reported.
General Surgery and Internal Navigation
While plastic surgery often deals with the body surface, general surgeons are using AR to navigate mobile organs.
-
Stoma Creation: In a first series of 14 patients at Weill Cornell, surgeons used a HoloLens 2 headset to overlay a 3D map of the bowel, bones and vessels and mark the path to the target loop on the skin; the right loop was found in every case, without open surgery [2].
-
Laparoscopic Liver Surgery: Systems like “Smartliver” overlay 3D models of the liver and its blood vessels onto the laparoscopic video. In a UCL study with 13 patients, surgeons found the overlay intuitive and potentially helpful for decisions, but it was still aligned by hand; automating the alignment and predicting how the liver deforms are the next steps [5].
-
Minimalist Setup: In the first US series of minimally invasive operations with the Apple Vision Pro (41 cases at UC San Diego), surgeons used up to three virtual displays of the laparoscopic or endoscopic video, placed where they wanted in their field of view, instead of wall monitors; they reported a low workload and there were no complications, though the study had no comparison group [4].
This technological shift goes beyond the TV drama and represents a move toward “GPS for the human body.”
Systems like ‘Smartliver’ overlay 3D models of the liver’s blood vessels onto the laparoscopic video. In a first study with 13 patients, surgeons found the overlay intuitive and potentially helpful, but alignment was still manual and keeping up with the liver’s movements remains a challenge.
In a first series of 41 operations, the headset showed up to three virtual screens of the laparoscopic video wherever the surgeon wanted them, so they did not have to look away at wall monitors, and surgeons reported a low workload. The study had no comparison group, so benefits over standard monitors are not yet proven.
Virtual Reality in Surgical Training
The impact of VR varies across specialties, but the most significant gains are seen in high-precision fields like orthopedics and ophthalmology.
1. Orthopedic Surgery
Orthopedics requires a deep understanding of 3D spatial relationships and haptic precision. A 2025 meta-analysis of 23 randomized trials (1,091 participants) in the Journal of Medical Internet Research found that VR-based education gave higher knowledge, clinical operation and surgical design scores than traditional teaching, though the trials varied widely [6].
2. Ophthalmology
In eye microsurgery the evidence is strongest: a 2025 meta-analysis of 17,623 cataract operations found that posterior capsular rupture (PCR), an important complication, was less frequent after trainees practiced on the Eyesi simulator (odds ratio 0.71) [7].
3. Plastic and Reconstructive Surgery
Plastic surgery often involves intricate “what-if” scenarios. VR allows residents to visualize patient-specific anatomy in 3D. Published evidence for plastic surgery training is thinner than for orthopedics and eye surgery.
In a 2025 meta-analysis of 23 randomized trials, VR teaching gave higher knowledge, clinical operation and surgical design scores than traditional teaching, and more interest and satisfaction, though the trials varied widely.
In a 2025 meta-analysis of 17,623 cataract operations, posterior capsular rupture was less frequent after trainees practiced on a simulator (odds ratio 0.71), and 22 of the 25 studies reviewed found an overall benefit from XR training.
VR lets trainees rehearse procedures and study 3D anatomy before a case, but the strongest published evidence so far comes from orthopedic and eye surgery training.
Challenges and Limitations
| Feature | HMD (Headsets) | Projected AR (PAR) |
|---|---|---|
| Immersion | High (3D Holograms) | Moderate (2D/3D Surface) |
| Ergonomics | Possible Neck Strain | No Extra Weight |
| Team Access | Individual User | Visible to Entire Team |
Despite the benefits, AR is not yet standard in every hospital. Key hurdles include:
-
Soft Tissue Deformation: Unlike bones, internal organs move when a patient breathes or when a surgeon touches them. Maintaining the “alignment” of the hologram to a moving organ is a major technical challenge [5].
-
Ergonomics: Head-mounted displays have a restricted field of view, can be uncomfortable to wear and show the overlay to one user only. Projected AR (PAR) instead projects markings directly onto the patient, with no headset and visible to the whole team [1].
Understanding these advanced tools can help demystify the experience for those preparing for a procedure.
Internal organs move and change shape when a patient breathes or when a surgeon touches them. Keeping a static 3D hologram perfectly aligned with an organ that is constantly shifting remains one of the primary technical hurdles for AR technology.
Yes, Projected Augmented Reality (PAR) is an alternative that beams surgical data and markings directly onto the patient’s skin using overhead projectors. This eliminates the need for heavy headsets and allows the entire surgical team to see the data simultaneously.
Summary of Key Takeaways
-
Real-Time Data Integration: AR overlays 3D patient anatomy (from CT/MRI) directly onto the surgical site, allowing surgeons to “see through” skin and tissue.
-
Eliminating Distraction: Headsets like the Apple Vision Pro let surgeons see the camera video without looking away at wall monitors; in a first series, surgeons reported a low workload [4].
-
Precision in Reconstruction: In plastic surgery, AR is being tested to locate blood vessels and plan incisions; early reports are small and outcome benefits are not yet proven [3].
-
Collaborative Visibility: Projected AR (PAR) allows the entire surgical team to see the guides simultaneously without requiring individual headsets [1].
Action Plan for Patients and Practitioners:
- For Patients: If undergoing complex reconstructive or abdominal surgery, ask your surgeon if they utilize “image-guided navigation” or AR for preoperative planning.
- For Surgeons: Evaluate the ergonomic trade-offs between Head-Mounted Displays (maximum immersion) and Projected AR (better team collaboration/comfort).
- Integration: Ensure preoperative imaging is high-resolution (thin-slice CT/MRI) to maximize the accuracy of the AR hologram registration.
Augmented reality could move surgery from mental mapping toward data-guided navigation, but most of the systems described here are still in small early studies.
| Benefit Category | Impact on Surgical Outcome |
|---|---|
| Precision | Real-time visualization of sub-surface anatomy and blood vessels. |
| Cognitive Load | Reduces the need for manual mental mapping from 2D scans. |
| Efficiency | Expected to shorten the search for structures like perforators; not yet shown in large studies. |
| Safety | Aims to help avoid critical structures; liver systems are at an early clinical stage. |
Not yet. While it is transforming surgery into a more data-driven science, it faces challenges like equipment costs, ergonomic concerns, and the technical difficulty of mapping moving internal tissues.
Patients undergoing complex abdominal or reconstructive procedures should ask if the surgical team utilizes ‘image-guided navigation’ or AR for preoperative planning to improve precision and safety.
Sources
-
[1] Projected Augmented Reality in Surgery: History and Validation – MDPI
-
[2] AR with Image Fusion for Surgical Stoma Creation – Frontiers in Virtual Reality
-
[4] AR Headsets in Minimally Invasive General Surgery – Surgical Endoscopy
-
[5] Assessing AR Displays in Laparoscopic Liver Surgery – Springer
-
[7] Nature: Extended reality simulators on ophthalmic surgical training