Virtual and Augmented Reality as Game-Changers in Anesthesiology: History, Applications, and Challenges for Clinical Virtuality
Angela N Johnson1,2*
1Department of Technical Communication, Texas Tech University, USA
2CTI Clinical Trials and Consulting, USA
Submission: December 19, 2017; Published: January 10, 2017
*Corresponding author: Angela N Johnson, Department of Technical Communication, Science and Healthcare, Texas Tech University, 2500 Broadway Lubbock, Texas 79409, USA, Tel: 910-540-9890; Email: angela.johnson@ttu.edu
How to cite this article: Angela N J. Virtual and Augmented Reality as Game-Changers in Anesthesiology: History, Applications, and Challenges for Clinical Virtuality. J Anest & Inten Care Med. 2018; 5(1) : 555652. DOI: 10.19080/JAICM.2018.05.555652
Abstract
Medical virtual reality (MVR) and medical augmented reality (MAR) systems are types of mixed reality technologies that are poised to dramatically alter anesthesiology practice and training. Virtual reality typically refers to placing a real person in a virtual environment; and is useful in applications like procedure simulation; tele-mentoring; training for complex block procedures and crisis management; and even therapies for neuropathic pain management. Augmented reality is a related technique where digital elements are over laid on the surface of real-world objects; useful in projecting landmarks based on ultrasound imaging; simplifying patient monitoring; and creating mixed reality simulations (often combining volunteers or dummies with digital elements) that enable simulations of crisis; transport; and emergency medicine situations in a low risk environment. Such technologies are already being used by military medical centers to improve care during air transport. Further; both MVR and MAR have numerous applications in medical school training and residency; enabling anesthesiology trainees to build psychomotor skills and get real-time feedback without posing risks to real patients. This review provides an update on the state of MVR and MAR technologies relevant to anesthesiology. It also provides insights into technology trends; such as telemedicine and robotics; that will likely have a dramatic impact on anesthesiology practice over the coming decade.
Keywords: Virtual reality; Augmented reality; Anesthesia delivery; Local anesthesia; Tele surgery; Digital health; Simulation; Virtuality; Mixed reality
Abbreviations: AREA: Augmented Reality System for Epidural Anesthesia; CASE: Comprehensive Anesthesia Simulation Environment; CBCT: Cone Beam Computed Tomography; CRPS: Complex Regional Pain Syndrome; CT: Computed Tomography; HIT: University of Birmingham's Human Interface Technology; HMD: Head-Mounted Displays; MAR: Medical Augmented Reality; MERT: UK Armed Forces' Medical Emergency Response Team; MMR: Medical Mixed Reality; MVF: Mirror Visual Feedback; MVR: Medical Virtual Reality; PCT: Perceptual Control Theory; PLP: Phantom Limb Pain; RV: Reality-Virtuality; SAT: Self-Affirmation Theory; SSH: Society for Simulation in Healthcare; TTMBC: Trans Theoretical Model of Behavior Change; VAM: Virtual Anesthesia Machine
Introduction
Medical augmented reality (MAR) and medical virtual reality (MVR) simulations are posed to play an important role in the future of anesthesiology training and practice. These technologies have already appeared in medical school and residency training programs; providing new ways for trainees to develop complex motor and critical thinking skills requires for complex procedures like nerve block. Both virtual reality and augmented reality have applications in anesthesiology; but these terms should not be confused. Virtual reality is typically considered the placement of an individual into a synthetic or unreal environment; whereas augmented reality is a somewhat newer technology that adds virtual "layers" onto existing real-time experiences [1].
In 1994; Milgram's landmark work in virtuality suggested that augmented reality could be considered a reality-virtuality (RV) continuum based on the ratio of real-world and virtual elements in mixed realities [2]. This continuum can readily be adapted to the case of anesthesiology practice in medicine; with a spectrum of medical mixed reality (MMR) ranging from real environment procedures to completely simulated virtual procedures (Figure 1) . In between these extremes; we encounter medical augmented reality (MAR) procedures like projected land marking and eye-glass patient monitors; and medical virtual reality (MVR) simulations and surgical guides. Armed with this terminology; anesthesiologists can begin to assess the use of mixed realities in their clinical practice; and those engaged in quality systems and care pathway planning can begin to consider mixed realities as a viable clinical solution. This brief review is designed to give clinicians and administrators a common language to discuss mixed reality developments in the clinic. It will also overview promising technologies relevant to anesthesia practice that have recently emerged.
Overview of Virtual and Augmented Reality in Anesthesiology
Origins of Simulated Reality in Anesthesiology
Owing to the serious nature of adverse events that can occur in anesthesia practice; medical simulated reality (or simply "simulation") has played a rich role in the development of anesthesiology practice and training. Fostered by advancements in audiovisual technology in the 1980s; a Stanford University research team led by David Gaba developed an early anesthesia simulation model called the Comprehensive Anesthesia Simulation Environment (CASE) using a Macintosh computer; mannequin; and waveform generators [3]. The model was subsequently shown to give dramatic improvements in team-based training and anesthesia crisis management [4,5]. In 1989; Gaba and DeAnda also reported the use of videotaped simulations to help anesthesia trainees learn to identify and response to simulated critical events; including end bronchial intubation; kinked IV; a trial fibrillation with hypotension; breathing circuit disconnection; and cardiac arrest [5]. The results showed that simulated experience could help anesthesia trainees detect problems and improved the response pathway necessary to troubleshoot problems in the clinic.
Following development of these new technology-based methods for simulation; scholars of and ragogy; or the study of how adult learners acquire new skills; produced evidence showing conclusively that medical simulation by itself does not guarantee learning but help adults take on new skills more efficiently [6,7]. However; dummy simulation and video recording methods lack the ability to be highly customized for unique patient path physiology. They also cannot perfectly approximately the tactile experience of placing anesthesia in practice; leaving much to be learned by experience on volunteers or initial patients. Comparatively virtual and augmented reality is a media that can create a more effective simulation that ever before possible. This allows many training procedures to be done via highly realistic simulations; and may also enable training on crisis management in anesthesiology as never before possible.
Virtual Reality for Simulation in Anesthesiology
Medical virtual reality (MVR) is a type of mixed reality where real elements (or a person or clinician) are placed into predominantly simulated environments; such as typical virtual reality headsets which emerge the viewer in 3D simulation models or even simulated clinical procedures. These systems have many advantages; including allowing trainees to repeat new techniques in an environment that does not pose risk to patients or require human volunteers. Perhaps more importantly; virtual reality systems enable quantitative assessment of competency in both typical and high-risk scenarios. As these technologies have become increasingly widely available over the past decade; organizations like the Society for Simulation in Healthcare (SSH) founded in 2004 have emerged to promote the use; availability; and adoption of sophisticated simulation models [8].
Augmented Reality for Simulation in Anesthesiology
Augmented reality is a somewhat more modern concept than its predecessor virtual reality. The term augmented reality was first coined in 1992 by Tom Caudell; who described it as a mixed reality that combined virtual elements with the real world [9]. Examples of this have been seen in pop culture; such as the massively popular Pokémon Go mobile application; which overlays gaming elements on a real-world view and have been shown to have an impact on physical activity; perceptions; and other health factors [10]. In medicine; augmented reality has been used to display patient monitoring data; eliminating the need for the anesthesiologist to interrupt a procedure by looking from procedure to patient monitor; thus enabling more focus on the procedure [11].
Why does mixed-reality work in medicine?
While many theories of learning interpret medical mixed reality differently; perceptual control theory (PCT) is popular theory is that can explain the effectiveness of virtual and augmented reality for clinical use. PCT holds that change is a contextual process based on exploration and control; where we are in a constant process of exploring our environment and seeking to bring what we observed and want into alignment (or control) [12,13]. Other theories; such as self-affirmation theory (SAT) and the Trans Theoretical Model of Behavior Change (TTMBH) further explore how virtual reality can be a more effective means of enacting "personal change;" which can mean learning new skills in anesthesiology more effectively [12]. These theories largely argue that virtual and augmented reality offer a transformative experience; much like reality; triggering behavioral; emotional; and intellectual associations that manifest when a person goes to repeat a task first encountered in a mixed reality. This type of experiential learning may revolutionize training for anesthesiology as well as offering new opportunities to apply experiential learning to complex or challenging procedures before performing them on the real patient; thus limiting discomfort and risk of injury.
Applications of Augmented Reality in Anesthesiology
Medical augmented reality (MAR) involves the overlay of digital elements on existing real 3D structures; such as the body. This may involve projection onto a surface (such as the skin) or viewing through a monitor; transparent glass; or head-mounted display visible to the clinician [14]. There are numerous current examples of these devices in anesthesiology; many of which are already commercially available though adoption has been relatively slow.
Training
Training for complex nerve block procedures is a central application of virtual reality that is gaining traction in practice and appearing at medical and dental schools worldwide. The benefit of virtual reality for these procedures is that anesthesiology trainees can examine real anatomy and repeat procedure; all while receiving feedback in a simulated environment. In particular; virtual reality training simulators have become available for inferior alveolar nerve block; a procedure commonly performed in dentistry [15]. Other simulations are also available in increasing numbers as technologies and adoption increased. While virtual reality has clear applications in simulation; augmented reality has a special niche ability to engage psychomotor and visualization skills by combining real elements (such as volunteers and mannequins or dummies) with visual overlays; which have the potential to offer a very highly realistic learning experience that has been demonstrated to support complex medical learning and transfer skills. This is of particular important to complex psychomotor skills; such as those required to perform block and sensitive needle procedures in anesthesiology.
Patient Monitoring
One of the most promising areas of augmented reality for medicine is in patient monitoring; where headsets and other devices can enable the doctor to visualize patient monitoring data without looking away from the procedure at hand [11] (Figure 2) . For any anesthesiologists who has had to twist to look at a monitor; this could dramatically improve procedures. Because gaze angle can easily impact needle progression or insertion angle; this has the potential to improve many procedures. However; much of the information needed by anesthesiologists; such as ultrasound displays; has not conventionally been available in augmented reality patient monitoring equipment. Recently; researchers from the University of Toronto reported success using a novel augmented reality system designed specifically for anesthesiologists; which is capable of displaying routine vitals as well as ultrasound data [16].
In 2007; a pilot study of remote administration of an EEG-controlled closed loop administration of propofol (a teletherapeutic propofol infusion) was performed in 11 patients undergoing general surgery with good outcomes. Results demonstrated that tele-therapeutic drug delivery was feasible in anesthesia; but required careful monitoring of vitals onsite [17]. The same technology that is used for local augmented reality patient monitoring may have the ability to improve current tele-therapeutic drug delivery options by enabling a remote anesthesiologist to access robust patient monitoring and imaging data during drug delivery. This could be particularly beneficial for critical care patients in remote areas.
Epidural Analgesia
Epidural analgesia is a common practice in obstetrics as part of labor and cesarean delivery; as well as certain other surgical interventions where they serve as viable alternatives to general anesthesia. Lumbar detection is critical to placement of epidural needles and effective resultant nerve block; and it most commonly performed unassisted via manual palpation which has only about a 30% success rateor using =ionizing radiation like CT or X-ray [18]. In 2012; Ashabet et al. [18] reported an augmented reality system which produced an overlay of lumber levels digitally based on ultrasonic images called the Augmented Reality System for Epidural Anesthesia (AREA) platform; with a success rate of 85% for lumbar vertebrae identification.
Oral; Maxillofacial; and Dental Anesthesia
Augmented reality has been used extensively in oral and maxillofacial surgery and dentistry and more recently to achieve alveolar nerve block; a common anesthesia procedure performed in dental clinics [19-21].
Applications of Virtual Reality in Anesthesiology
Training
Many applications of medical virtual reality already are used in medical school and residency programs; including for use in anesthesia training. In 2014; Dr. Shafi Ahmed; co-founder of the VR company Medical Realities; and his team of doctors in London performed the world's first 360-degree live stream VR surgery; available to audiences worldwide via the Medical Realities website [1]. Over 13;000 clinicians and residents were able to observe the surgery in real-time using virtual reality; enabling a highly realistic observation experience with 360-degree visualization of the operating theatre. This technology has the potential to turn any clinical procedure into a virtual training environment for real-time or future use. Virtual reality can also be used to simulate certain procedure; particularly those difficult to replicate in the clinic. Researchers at the University of Birmingham's Human Interface Technology (HIT) group are working with the military; in particular the medics of the UK Armed Forces' Medical Emergency Response Team (MERT); to develop a simulator that allows medical procedures to be replicated in a virtual simulation set in the inside of a Chinook helicopter [22]. The goal is to train military medics to respond in emergency medicine and air transport situations which would be virtually impossible to replicate without virtual reality technology.
Novel applications have also been developed to help anesthesiologists learn more about the equipment they use on a daily basis; which can improve their response in a crisis or device malfunction situation. The Virtual Anesthesia Machine (VAM); more commonly known as the "glass" anesthesia machine (http://vam.anest.ufl.edu/simulations/vam.php]; is a web-based simulation model developed by researchers at the University of Florida that enables the users to see the inner working of an anesthesia machine [23]. Initial research showed that training using equipment virtualizations improved physician engagement and understanding of these devices; though further work will be necessary to determine if there is a positive impact on crisis management. With devices like Gear VR; VR One; and Google Cardboard; hospitals are changing their approach to personnel training rapidly; and institutions like Miami Children's Hospital have reported an unexpected additional benefit of virtual reality training via increased employee retention rate; as high as 80% with virtual reality training programs compared to only 20% with conventional training [1].
Pain Management
Both virtual and augmented reality has clear medical applications in physical therapy; but more recently has been employed in pain management. Researchers from the University of Brussels and Erasmus Hospital's Department of Anesthesiology in Belgium developed a treatment for neuropathic pain in which a patient wearing 3D glasses observed themselves on a screen; where the user can interact with various objects. This therapy was used to control phantom limb pain (PLP) and complex regional pain syndrome (CRPS) [24]; providing a therapeutic function similar in principle to mirror visual feedback (MVF) which is often used to restore normal neuropathic function in unilateral in amputation patients [25]. Conventional MVF has also been performed with virtual reality with good results for pain relief [26]. These therapies work by enabling patients to interact with virtual objects in the augmented reality environment (Figure 2). Given the positive effect of visual feedback when a mirror is used; it is perhaps unsurprising that virtual reality has clear applications in management of PLP and CRPS; which rely highly on retraining the body using visual and tactile stimuli.
Surgery and Telemedicine
Virtual reality systems may also open the door to future "tele" surgery options; including telementoring; telemonitoring; teleproctoring; and robotic telesurgery [27]. It is quite feasible that the same technology currently used to power augmented reality patient monitoring systems could be coupled with robotic surgery; allowing anesthesiology procedures to be performed in remote or controlled environments. In fact; early feasibility trials showed success with remote robot-assisted telesurgery as early as 2002 [28]. Telemedicine applications have already been used clinically for anesthesiology consultation; home monitoring; offsite (remote) critical care monitoring; and tele-therapeutic drug administration [29]. As telemedicine is becomes more readily accessible and affordable and continues to grow rapidly with promising results; virtual reality technologies are a clear fit to improve and enhance telemedicine technologies.
Dental Anesthesia
Wide adoption of cone beam computed tomography (CBCT) has increased the use CT in the oral and maxillofacial fields; and 3D simulations generated from CT data are becoming commonplace; both with and without the use of virtual reality display platforms [30]. These simulations can be used to explore actual anatomy prior to highly varied procedures like jaw replacement and implant placement; and they enable precise localization of nerves in the face and jaw preventing possible injury. This is particularly important as much as 0.5 to 7.5% of molar surgeries with anesthesia involve nerve damage or disruption (primarily to the lingual nerve and inferior dental nerve) which may be related to anesthesia delivery during procedures [31]. Further the rates of iatrogenic lingual nerve injuries (primarily inferior alveolar and trigeminal nerves) following prosthetic implants and endodontic may occur in up to 30% of patients; and as much as 17% of these serve injuries may be related to anesthesia delivery [32]. Virtual reality and 3D modeling of real maxofacial anatomy has the potential to help reduce anesthesia-related dental procedure injuries.
Challenges Facing Virtual Reality in Anesthesiology
A central challenge for the application of augmented reality systems; in particular for land-marking; is patient movement. Augmented systems may be mounted to the patient's body or have algorithmic adjustments built into the software that correct for motion; but currently there is no standard protocol for handling movement [33,34]. In anesthesia; the most common solution is to use head-mounted displays (HMDs) that overlap computer simulations and the real field of vision [9,35] (Figure 3). Several promising initial studies utilized HDM to simulate adverse event detection in anesthesia; with mixed results in early studies [36] and promising results in more recent trials [37]. Another challenge facings adoption of MAR and MVR systems is economic. Falling hardware prices have bolstered clinical purchases ofthese systems; with systems like Oculus Rift Headset retailing at about $599; while the higher end HTC Vive costs about $799 [38]. In addition; powerful workstations; software; and peripherals can range from a few hundred to tens of thousands of dollars; particularly when those systems integrate with existing equipment like monitors; printers; medical records systems; and 3D printing or scanning systems. Further; most clinics underestimate the amount that must be invested in user training to bring initial users up to speed. While tech-savvy research centers often have technical specialists able to setup such equipment; adoption at primarily clinical facilities is somewhat more challenging and relies on a combination of expensive training options and championing from trained internal users. As these systems become more oriented on the average clinical user; accessibility can be expected to increase rapidly over the next few years.
Conclusion
Although the use of medical virtual reality and augmented reality in anesthesiology are emerging fields with clear challenges; current applications of these technologies present a starting point for integration into anesthesiology practice of the future. MVR and MAR systems have the potential to both provide useful training and surgical planning simulations as well as novel methods for highly accurate real-time operative guidance and potentially even remote procedures. Virtual and augmented reality technologies will certainly play a role in future anesthesiology practice as the technology continues to evolve; and tech-savvy users will be able to leverage these to accomplish increasingly sophisticated advances in care; ranging from remote therapeutics to augmented reality for precision needle placement. Further; it is increasingly important for anesthesiologists to learn the language of mixed reality technologies which are poised to change both training and practice in the next decade.
References
- T Foley (2016) Practical Applications for Virtual Reality in Healthcare.
- P Milgram, H Takemura, A Utsumi F, Kishino (1994) Augmented Reality: A class of displays on the reality-virtuality continuum. Telemanipulator and Telepresence Technologies 2351: 2351-2334.
- Gaba DM, DeAnda A (1988) A comprehensive anesthesia simulation environment: re-creating the operating room for research and training. Anesthesiology 69(3): 387-394.
- D Gaba, T Lee (1990) Measuring the workload of anesthesiologist. Anesth Analg 71(4): 354-361.
- D Gaba, A DeAnda (1989) The response of anesthesia trainees to simulated critical incidents. Anesth Analg 68(4): 444-451.
- D Gaba (2004) The future vision of simulation in health care. Qual Saf Health Care 13(1): i2-i10.
- F Jones, C Passos Neto, O Braghiroli (2015) Simulation in Medical Education: Brief history and methodology. Principles and Practice of Clinical Research 2(1): 56-68.
- (2017) Society of Simulation in Healthcare.
- A Del CastilloI, N SardilI (2012) Augmented reality: can it enrich the practice of anesthesiology? Rev Bras Anestesiol 62(6): 887-888.
- T Althoff, R White, E Horvitz (2016) Influence of Pokémon Go on Physical Activity: Study and Implications. J Med Internet Res 18(12): e315.
- D Ormerod, B Ross, A Naluai-Cecchini (2003) Use of an augmented reality display of patient monitoring data to enhance anesthesiologists' response to abnormal clinical events. Stud Health Technol Inform 94: 248-250.
- G Riva, R Bańos, C Botella, F Mantovani, A Gaggioli (2016) Transforming Experience: The Potential of Augmented Reality and Virtual Reality for Enhancing Personal and Clinical Change. Frontiers in Psychiatry 7: 164.
- J Vancouver, D Putka (2000) Analyzing goal-striving processes and a test of the generalizability of perceptual control theory. Organ Behav Hum Decis Process 82(2): 334-362.
- G Badiali, V Ferrari, F Cutolo, C Freschi, D Caramella, A Bianchi, et al. Augmented reality as an aid in maxillofacial surgery: validation of a wearable system allowing maxillary repositioning. J Craniomaxillofac Surg 42(8): 1970-1976.
- C Corręa, M Machado, E Ranzini, R Tori, F Nunes (2017) Virtual Reality simulator for dental anesthesia training in the inferior alveolar nerve block. J Appl Oral Sci 25(4): 357-366.
- (2017) Anesthesiology News. Novel Augments Reality Device Displays Visual Data Array.
- H Ihmsen, K Naguib, G Schneider, H Schwilden, J Schüttler, et al. (2007) Teletherapeutic drug administration by long distance closed-loop control of propofol. Br J Anaesth 98(2): 189-195.
- H Ashab, V Lessoway, S Khallaghia, A Cheng, R Rohling, et al. (2012) AREA: An Augmented Reality system for Epidural Anaesthesia. Conf Proc IEEE Eng Med Biol Soc 2012: 2659-2663.
- Y Won, S Kang (2017) Application of augmented reality for inferior alveolar nerve block anesthesia: A technical note. J Dent Anesth Pain Med 17(2): 129-134.
- M Zhu, F Liu, G Chai, J Pan, T Jiang, et al. (2017) A novel augmented reality system for displaying inferior alveolar nerve bundles in maxillofacial surgery. Sci Rep 7: 42365.
- J Wang, H Suenaga, K Hoshi, L Yang, E Kobayashi, et al. (2014) Augmented reality navigation with automatic marker-free image registration using 3-D image overlay for dental surgery. IEEE Trans Biomed Eng 61(4): 1295-1304.
- (2016) Virtual reality 'Chinook' to help train medics in UK Armed Forces. Phys Org.
- S Lampotang, D Lizdas, E Liem, W Dobbins (2007) The University of Florida, Department of Anesthesiology.
- B Penelle, D Mouraux, E Brassinne, T Tuna, A Nonclercq, et al. (2012) 3D augmented reality applied to the treatment of neuropathic pain. Proc. 9th Intl Conf. Disability, Virtual Reality & Associated Technologies, Laval, France, p. 1-8.
- C McCabe (2011) Mirror visual feedback therapy. A practical approach. J Hand Ther 24( 2): 170-178.
- D Mouraux, E Brassinne, S Sobczak, A Nonclercq, N Warzee (2017) 3D augmented reality mirror visual feedback therapy applied to the treatment of persistent, unilateral upper extremity neuropathic pain: a preliminary study. J Man Manip Ther 25(3): 137-143.
- P Gambadauro, R Torrejon (2013) The "tele" factor in surgery today and tomorrow: implications for surgical training and education. Surg Today 43(2): 115-122.
- J Marescaux, J Leroy, F Rubino, M Smith, M Vix, et al. (2002) Transcontinental Robot-Assisted Remote Telesurgery: Feasibility and Potential Applications. Ann Surg 235(4): 487-492.
- C Veena, J Attri, R Chatrath (2010) Telemedicine and anaesthesia. Indian J Anaesth 54(3): 199-204.
- Y Hernandez, B Tarazona, N Zamora, R Cibrian, J Gandia, et al. (2015) Comparative study of reproducibility and accuracy in measuring mesiodistal tooth sizes using three different methods: 2D digital, 3D CBCT, and 3D CBCT segmented. Oral Radiol 31(3): 165-172.
- K Rehman, K Webster, M Dover (2002) Links between anaesthetic modality and nerve damage during lower third molar surgery. British Dental Journal 193(1): 43-45.
- T Renton, Z Yilmaz (2011) Profiling of patients presenting with posttraumatic neuropathy of the trigeminal nerve. J Orofac Pain 254(4): 333-344.
- H Suenaga, TH Hoang, H Liao, K Masamune, T Dohi, et al. (2013) Realtime in situ three-dimensional integral videography and surgical navigation using augmented reality: a pilot study. Int J Oral Sci 5(2): 98-102.
- Y Won, S Kang (2017) Application of augmented reality for inferior alveolar nerve block anesthesia: A technical note. J Dent Anesth Pain Med 172: 129-134.
- D Liu, S Jenkins, P Sanderson (2009) Patient Monitoring with head- mounted displays. Curr Opin Anaesthesiol 22(6): 796-803.
- P Sanderson, M Watson, W Russell, S Jenkins, D Liu, et al. (2008) Advanced auditory displays and head-mounted displays: advantages and disadvantages for monitoring by the distracted anesthesiologist. Anesth Analg 1066: 1787-1797.
- D Liu, S Jenkins, P Sanderson, P Fabian, W Russell (2010) Monitoring with head-mounted displays in general anesthesia: a clinical evaluation in the operating room. Anesth Analg 110(4): 1032-1038.
- C Kern (2016) Dropping Virtual Reality Prices Yields Better Opportunities for Healthcare.