Automatic performance evaluation in surgical simulation

• Authors: Kenneth Salisbury;Christopher Sewell
• Venue: Automatic performance evaluation in surgical simulation
• Year: 2007

One of the most important advantages of computer simulators for surgical training is the opportunity they afford for independent learning. However, if the simulator does not provide useful instructional feedback to the user, this advantage is significantly blunted by the need for an instructor to supervise and tutor the trainee while using the simulator. In fact, the continued need for instructor feedback with most existing simulators is often cited as a primary reason for the reluctance of many medical schools to fully embrace simulator technology. Thus, the incorporation of relevant, intuitive metrics---and, just as importantly, the presentation of such metrics to the user in such a way so as to provide constructive feedback that facilitates independent learning and improvement---is essential to the development of efficient simulators. This dissertation presents a number of novel metrics for the automated evaluation of surgical technique. Although many of the concepts behind these metrics have wide application throughout surgery, they have been implemented specifically in the context of a simulation of mastoidectomy, a surgical procedure that involves drilling away part of the temporal bone in order to access the inner ear. First, the visuohaptic simulator itself is described, followed by the details of a wide variety of metrics designed to assess the user's ability to make correct decisions in response to discrete events in a surgical scenario, maintain proper visibility of the surgical field, achieve sufficient exposure of critical anatomic structures, remove the optimal volume of bone, apply appropriate forces and velocities as vulnerable structures are approached, and demonstrate expert technique in the positioning and movement of the drill and suction. Mechanisms for providing visualizations and other feedback to the user based on these metrics, both interactively in the simulator while performing the virtual procedure, and afterwards in an automated debriefing session using a novel performance evaluation console, are then elaborated. Next, the results of several user studies are reported, providing some preliminary validation of the simulator, the metrics, the feedback mechanisms, and the hypothesis that skills learned on a haptic simulator can transfer to a real-world, surgically-relevant task. Finally, our efforts to apply machine learning algorithms to our simulator data in order to automatically differentiate users' expertise levels are recounted, and some final thoughts are offered.