An AI system for evaluating pass fail in fundamentals of laparoscopic surgery from live video in realtime with performative feedback

Yunzhe Xue, Andrew Hu, Rohit Muralidhar, Justin W. Ady, Advaith Bongu, Usman Roshan

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Medical students preparing to be a surgeon are required to demonstrate proficiency in laparoscopic surgery as part of their training. This is done via simulation on a Fundamentals of Laparoscopic Surgery (FLS) kit where the student has to use graspers to transfer six rings from one set of pegs to another and then again back to the original set of pegs without dropping the peg. A peg dropped outside the box is considered a fail but a drop inside is the box is allowed as long as the transfer resumes with the correct grasper. We present an AI system that automatically determines if a student has passed or failed the FLS test. Our system uses an underlying YOLO model to detect the FLS box, the left and right graspers, and the FLS pegs and rings. We then use logic on top of this to detect events such as pick peg from ring, transfer peg between grapsers, and place peg on ring. We are also able to detect if the grasper drops the peg inside or outside the box (the latter being an automatic fail) and if the dropped peg was picked with the correct or wrong grasper. Our system detects these events in realtime without looking into future frames of the video - this means it can give performative feedback to the student as they are performing the task. To evaluate our system we trained it on 6 videos of junior medical residents performing FLS containing several instances of dropping pegs plus 1 video of a fake FLS showing deliberate drops across the board - this is so that the model can learn to identify drops. To evaluate our model we tested it on 14 videos on which our model correctly predicted pass fail on 11 - giving an accuracy of 78.6%. Compared to previous work our system requires only one camera, detects drops inside and outside the FLS box, produces a fully automated pass fail determination, and gives live performative feedback as the student is performing the task - thus informing them of their mistakes in realtime.

Original languageEnglish (US)
Title of host publicationProceedings - 2023 2023 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2023
EditorsXingpeng Jiang, Haiying Wang, Reda Alhajj, Xiaohua Hu, Felix Engel, Mufti Mahmud, Nadia Pisanti, Xuefeng Cui, Hong Song
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages4167-4171
Number of pages5
ISBN (Electronic)9798350337488
DOIs
StatePublished - 2023
Event2023 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2023 - Istanbul, Turkey
Duration: Dec 5 2023Dec 8 2023

Publication series

NameProceedings - 2023 2023 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2023

Conference

Conference2023 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2023
Country/TerritoryTurkey
CityIstanbul
Period12/5/2312/8/23

All Science Journal Classification (ASJC) codes

  • Artificial Intelligence
  • Computer Science Applications
  • Computer Vision and Pattern Recognition
  • Automotive Engineering
  • Modeling and Simulation
  • Health Informatics

Keywords

  • automatic evaluation
  • fundamentals of laparoscopic surgery
  • video AI
  • YOLO

Fingerprint

Dive into the research topics of 'An AI system for evaluating pass fail in fundamentals of laparoscopic surgery from live video in realtime with performative feedback'. Together they form a unique fingerprint.

Cite this