Augmented reality anatomy visualization for surgery assistance with hololens: AR surgery assistance with hololens

Enrique Castelan, Margarita Vinnikov, Xianlian Alex Zhou

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Immediate care for trauma wounded patients in austere or remote settings makes medical knowledge, skills, and efficiency of the on-duty medical professional paramount. For wounds that extend deep into internal anatomy, proper visualization of internal anatomy can enable more efficient and effective evaluation when presented to medical providers positioned close to the point of injury (POI). In this paper, a conceptual Augmented Reality (AR) surgical tool is presented to provide visualization of internal human anatomy, superimposed on the view of a patient, to assist medical providers for immediate casualty care. This AR surgical tool can play a role in 3D surgery or treatment planning as a navigational aid in preparing medical interventions and enhancing surgery or treatment procedures by displaying otherwise obscured anatomy and nearby vessels. Critical software and hardware components are integrated to construct a prototype AR system for the portable AR surgical visualization tool. The system uses a Microsoft HoloLens 1 and an Azure Kinect camera for simultaneous body-tracking and anatomy overlay to demonstrate the overall concept. Future extension of this work will aim to create a more accurate and compact prototype system that utilizes HoloLens 2 with an embedded Kinect camera for laboratory and field tests of its use in surgery assistance. Such an AR tool can also serve as a training tool for medical caregivers, applied with a human subject or a medical manikin.

Original languageEnglish (US)
Title of host publicationIMX 2021 - Proceedings of the 2021 ACM International Conference on Interactive Media Experiences
PublisherAssociation for Computing Machinery, Inc
Pages329-331
Number of pages3
ISBN (Electronic)9781450383899
DOIs
StatePublished - Jun 21 2021
Event2021 ACM International Conference on Interactive Media eXperiences, IMX 2021 - Virtual, Online, United States
Duration: Jun 21 2021Jun 23 2021

Publication series

NameIMX 2021 - Proceedings of the 2021 ACM International Conference on Interactive Media Experiences

Conference

Conference2021 ACM International Conference on Interactive Media eXperiences, IMX 2021
Country/TerritoryUnited States
CityVirtual, Online
Period6/21/216/23/21

All Science Journal Classification (ASJC) codes

  • Computer Graphics and Computer-Aided Design
  • Human-Computer Interaction

Keywords

  • Anatomy visualization
  • HoloLens
  • Pose estimation
  • Surgery assistance

Fingerprint

Dive into the research topics of 'Augmented reality anatomy visualization for surgery assistance with hololens: AR surgery assistance with hololens'. Together they form a unique fingerprint.

Cite this