A twenty-something woman in scrubs stands alone and articulates her hands in terse, repetitive motions. An insectile headset covers her upper face; her hand clutches a controller. Though her gesticulations suggest kinship with the local psychiatric ward or rehab facility, she’s actually a medical student performing a complex surgery—virtually.
Inside her headset, the student sees a three-dimensional rendering of an operating room complete with a patient, a surgical tray with tools, and other details pertinent to a real OR. She cuts, drills, bolts, sutures, and so on through the various stages of the surgery. She is the beneficiary of a new trend in medical training, one that allows her limitless opportunity to practice a procedure over and over, prior to ever touching a patient.
Virtual reality programs allow medical students to practice surgeries
At the University of Utah’s Spencer S. Eccles Health Sciences Library (EHSL), “we are currently doing a number of really interesting projects [with virtual technologies],” says Benjamin Engel, a U of U virtual reality programmer who runs the day-to-day operations at EHSL. His projects include a trauma lead simulation where a patient comes in with a number of issues into the trauma bay and the participant plays the role of the head doctor who must direct everyone to save the patient. The simulation allows students to “gain practice calling directions for an infinite amount of scenarios and cases."
EHSL also has a “really cool dental sim," says Engel, “that allows you to do just about everything a dentist would do but fully virtual, with a ton of data collection, and for much cheaper.” In both the trauma and the dental simulations, participants experience the respective scenarios via a headset that cuts them off visually from their environment and puts them in a fabricated one. Engels, and EHSL, is at the forefront of a new frontier in medicine.
According to “Virtual reality in healthcare" by J Mazurek, et al, virtual reality (VR) is “an image of artificial reality created entirely in three-dimensional graphics…" In other words, a computer simulation. The field of VR is also known as “phantomatics," a moniker that conveys its disconnect from the so-called “real" world. Indeed, many folks associate VR with a gaming subculture of unemployed 33-year-olds escaping their unfulfilled dreams in a digital trance amid empty pizza boxes in their parents’ basement.
But practical virtual reality technologies were originally developed, not as an escapist technology, but to serve a very practical purpose in the aerospace industry. VR simulators allow pilots-in-training to complete hundreds or thousands of landings and takeoffs before taking the controls of an actual aircraft. Which is good: I wouldn’t want to be a passenger during an absolute greenhorn pilot’s maiden landing, sans VR preparation.
And if pilots use VR to gain proficiency prior to putting lives on the line, why not surgeons?
With COVID-19 making in-person classes increasingly untenable, remote learning solutions are more germane than ever. “Instead of having hundreds of medical students sit in the classroom learning how to do surgery," says Han Jin. AR/VR technologies “will allow them to practice and learn from the comfort of their homes."
Jin is the CEO of Lucid, a startup platform that enables enterprises to build AI applications―but it also has a footing in 3D imaging. The company’s LucidPix smartphone app allows users to take 3D photos. While LucidPix is marketed as a recreational consumer product, Jin envisions his 3D imaging technology having healthcare applications.
“The number one reason 3D photos even exist,” Jin says, “is because of future uses for VR and AR." The virtual reality industry, he says, has difficulty creating fast and easy content for virtual simulations―3D photos were created to fill that gap.
Theoretically, surgeons could record an entire surgery in 3D with something like LucidPix and provide it to medical students to study remotely. Socially distanced learning is certainly advisable in the current era, but as it turns out, COVID or no, several compelling reasons exist for the integration of AR/VR into medical training.
Augmented reality programs give surgeons access to more data
If virtual reality is divorced from one’s surroundings, augmented reality (AR) provides a watered-down variation where one interacts with the real environment with mediation from virtual elements interpreted via a digital device such as a smartphone.
Jin explains that “the difference between AR and VR is the degree of immersion." With augmented reality, you can “overlay 3D content over the real world," whereas virtual reality “gives you the complete immersion into a different world."
Augmented reality experiences have been around a while. “If you’ve played Pokemon GO or used a Snapchat filter, you’ve gotten a taste of AR," explains Jack C in an article for EHSL. “Unlike VR, which requires you to strap a headset to your face, AR can be accomplished with nothing more than a phone or a computer."
Nivanthika K. Wimalasena, in Harvard University’s “Science in the News" blog, defines AR as, “a computerized projection of images or other information like scans, blueprints, or maps that is overlaid with, and anchored to, the physical world." Wimalasena foresees surgeons wearing AR glasses in the operating room that allow them to “visualize crucial information from scans overlaid with the body of the patient."
In other words, if you’re looking at a real patient, you’re also seeing their X-ray results, CAT-scan results, or MRI. Pretty exciting stuff. And, unlike a VR headset that blocks out the environment, AR glasses have clear lenses that allow us “to see the world in front of us, while mapping virtual information on top of it."
With AR/VR training—much like the pilot learning on the flight simulator prior to taking the controls of a jumbo jet—a surgical resident could perform surgery z, y, x, and a dozen more, hundreds of times each, before ever practicing on a patient. What’s more, the resident’s AR/VR training needn’t foreclose participating in a hands-on surgery—when the opportunity appears.
VR/AR programs provide continuing education opportunities for surgeons
New practitioners aside, even the most grizzled surgeon faces constant innovation within their field. In the old training model, residents learn on the job with more experienced surgeons. However, such an approach leaves much to chance: a resident might participate in 10 instances of surgery x, 2 of surgery y, and never encounter surgery z at all. As a future surgery z patient, do you want to be the guinea pig?
New medical devices, particularly, change a procedure and present a significant learning curve. Indeed, according to its 2019 “Research: How Virtual Reality Can Help Train Surgeons" Harvard Business Review states that surgeons may “travel to a one- to two-day training workshop with the medical device company," but then “it could be four to six months from the time surgeons train on a procedure to when they perform it on a patient."
Medical advances have outpaced the ability for surgeons-in-training to absorb all the complex surgeries and gain proficiency in them. Indeed, the Harvard Business Review report cites a University of Michigan study that showed “30 percent of surgeons couldn’t operate independently after residency." Of the 70 percent who purportedly could operate independently, it is doubtful they were proficient in every surgery within their field; there are just too many complex surgical procedures, with new ones pioneered regularly.
Augmented and virtual reality tech could allow surgeons to practice with a new device and refresh their knowledge as many times as needed.
But is AR/VR learning actually effective? As we learned from aircraft pilot simulations, they are. The Harvard Business Review report cited above reported that “VR training improved participants’ overall surgical performance by 230 percent compared with traditional training methods."
In the study, 20 participants were randomly assigned to “a traditionally trained group" and a virtual reality group that utilized the Osso VR platform. In addition to the 230 percent score increase (measured by the Global Assessment Five-Point Rating Scale), VR-trained participants were “an average of 20 percent faster" and also “completed 38 percent more steps correctly" as per a checklist for each procedure.
Back at EHSL, Engel dreams of the day he has more funding “to put more resources into our trauma sim." As cool as the current iteration is, he’d like to build in a generative patient situation that would dynamically adjust parameters “so you never see the exact same trauma patient twice." Practicing in unpredictable and unforeseen circumstances: that sounds suspiciously like the real thing.
“In the real world," Engel explains, “you need to adapt to a changing environment." And indeed, medicine itself is just such a changing environment with the proliferation of technological innovation—AR and VR included.