Proficiency-based progression process training for fundamentals of robotic surgery curriculum development

Introduction

As any new surgical device is commercialized, there is an obvious need for structured training. Shortly after robotic surgery emerged, there were a few robotic surgery skills training courses published, however they were by single institutions and focused upon local needs (1,2), with many of the courses using the skills module from the industry Instruction for Use (IFU) course (3). The Fundamentals of Robotic Surgery (FRS) is a standardized basic technical skills curriculum (course) which was developed by all 11 specialties performing robotic surgery and validated in a multi-institutional (14 institutions) trial (4). A report of the intended design was published in 2014 before completion of the consensus conferences, training and validation (5). This is a summary report now that the course is completed and validated. Rather than current industry-required course which is mandated by the FDA as a manual on how a medical device (robot) works, the FRS is a course that is designed by surgeons for surgeons to train and assess novice robotic surgeons on how to use the robot for clinical surgery. It is specifically limited to the basic psychomotor skills needed intra-operatively.

During the same time frame when the FRS began development, there was the introduction of both new surgical simulators, and more importantly, the Proficiency-based Progression (PBP) educational process (6-8). Unlike previous courses, this approach uses outcomes measures with metrics to assess learner performance in a standardized course, in which the passing benchmarks are based on the mean score of surgeons experienced in the use of robotics. The learner must achieve the proficiency ‘benchmark’ score regardless of amount of training time before being able to progress to advanced training and clinical surgery.

Methods

Following a small group session with subject matter experts (SME), a needs assessment process confirmed the necessity of starting from first principles using the Full Life Cycle Development method (FLCD) (9,10) and including all 11 surgical specialties that perform robotic surgery. The course was designed to be agnostic to both specialty and robot (even though only one robot was FDA approved). The course development began with a series of 4 consensus workshops, using the Delphi process (11): Outcomes Measures with Metrics, Curriculum Content Development, Creation of Simulation Models, and Validation Trial Design. Over the 2 years of the workshops, 80 different SMEs participated from multiple disciplines, including surgeons (official representatives of their specialty society), Surgical Society representatives, accreditation boards, federal regulators, engineers, medical educators, behavioral psychologists, etc. (Full list in 4).

The curriculum consists of the following, which contain figures, tables and appendices (9,10):

• The Outcomes Measures/Metrics were quantitative (checklists, binomial, etc.) and qualitative (unambiguous definitions on GEARS Likert scale), with an emphasis on avoidance of errors, which facilitated accurate assessment using formative feedback during the validation trial. The surgeon SMEs identified and defined 25 unique skills, which then were incorporated into 7 distinct tasks
• The Curriculum Content Development used the Proficiency-based progression (PBP) process and consisted of 2 components: (I) Didactic component, created as an initial web-based multi-media online course (12) with a post-test which must be passed with a 90% passing score (benchmark value based on experienced surgeons objectively assessed performance) before being allowed to progress to the technical skill portion and (II) Psychomotor skills portion, which required the creation of a new skills model, called the DOME (see below), which were determined by the outcomes measures/metrics. There were 25 skills, incorporated into 7 tasks, which required a new model because using the 2-D Fundamentals of Laparoscopic (FLS) model did not demonstrate evidence of construct validity when used on the robotic system—the extra degrees of freedom of the “wrist” of the robotic instruments allowed performance of tasks which were not able to be accomplished with straight laparoscopic instruments. Since this is a basic technical skills course, the content is limited to those technical skills used from entrance to the operating room until exit after the completion of the procedure—pre-operative and post-operative skills are not addressed.
• The Simulation Model is a 3-D, two layered ‘Dome’ model (Florida Hospital Nicholson Center, Celebration, Florida), the external layer of ‘skin’ was used for dissection/excision of a “puzzle piece”, under which was an “artery” to be dissected and excised (4). The engineers initially created the model in a computer-assisted-design/computer-assisted-manufacturing (CAD/CAM) software program, and exported the same software file to both a 3-D printer for a physical model, and for an identical virtual reality (VR) model, which was used on the two VR simulators: The dV-Trainer robotic surgery VR simulator (Mimic Technologies Inc, Seattle, WA) and the DVSS “da Vinci backpack” VR simulator (3-D Systems/Simbionix, Tel Aviv, Israel).
• The Validation Trial Design was a multi-institutional, multispecialty, single-blinded, parallel group randomized control trial (4). There were 11 surgical specialties from 14 institutions, which were required to be certified by the American College of Surgeons-Accredited Educational Institutes (ACS-AEI). There were 30 Experienced Surgeons to establish both the didactic and technical skills performance benchmarks. There were 4 groups randomized to, robot+ physical Dome, DVSS “Backpack”, DvTrainer, and the Control (which used the IFU industry model or slight modification thereof). In order to demonstrate that the FRS skills on the models translated to a clinically relevant identical model, 5 of the same tasks were designed to be performed upon a turkey leg, including the dissection and removal of a layer of skin and excision of an artery.

Results

Using the Full Life Cycle Development method to create a PBP course provides an effective and efficient process to create a surgical skills course which can accurately train and assess the technical performance of robotic basic surgical skills, as demonstrated by the example of the FRS curriculum (4). Both the primary outcomes measure (the course effectively trains and assesses the performance of novice surgeons) and secondary outcomes measures (I. equal training results with either a physical model or virtual reality simulator and II. PBP quantitative metrics are superior to qualitative Likert scale metrics for basic skills courses) were demonstrated. An additional benefit is that there are templates to facilitate rapid and effective standardized PBP skills course development which are freely available, open source material (10,12). Finally, pre-test/post-test assessment on a turkey leg model demonstrated transfer-of-training effectiveness in a clinically similar model.

Discussion

The FRS is a touchstone for next generation of basic technical skills course development. The use of simulation is a proven method for training novice surgeons in a safe environment, and is especially applicable for robotic surgery, where using a clinical robot for basic skills training decreases availability of the robot for patient surgical care. The FLCD method of simulation curriculum development, based upon modification of historical military methodologies of simulation, provides an established, standardized comprehensive approach. The PBP process represents a paradigm shift, in the terms that it permits accurate, quantitative measure of technical skills performance to the level of average experienced surgeons, thus providing objective evidence of novice attainment of technical skills. The primary use should be for fundamental technical psychomotor skills.

The rigorous methods and processes used in the FRS development and validation have been amply documented, and since it is an open source curriculum methodology, the templates that have been developed are freely available (12). Subsequent courses, such as the Robotic Training Network gynecologic robotic surgery course (13), have used the templates and have been able to decrease the cost and time to development of a course to less than 30% of the original FRS development cost.

Conclusions

The FCLD and PBP are proven curriculum development, training and assessment tools which have been used for effective training basic technical skills for robotic surgery. An exemplar is the FRS standardized course, which has demonstrated effectiveness of using PBP quantitative assessment of performance to proficiency. We advocate for all surgical specialties’ training programs for robotic surgery should implement the basic technical FRS skills before progressing to advanced (full procedure) training and only then to clinical robotic surgery. In addition, all future surgery course development could utilize the same PBP process, which can be facilitated by the open source templates.

Acknowledgments

Funding: None.

Footnote

Provenance and Peer Review: This article was commissioned by the Guest Editors (Steven D. Schwaitzberg and Rafael Perez) for the series “Advances in Robotic Surgery” published in Annals of Laparoscopic and Endoscopic Surgery. The article has undergone external peer review.

Conflicts of Interest: Both authors have completed the ICMJE uniform disclosure form (available at http://dx.doi.org/10.21037/ales.2020.02.04). The series “Advances in Robotic Surgery” was commissioned by the editorial office without any funding or sponsorship. The authors have no other conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. There were no patients or participants involved in this study, and therefore qualifies for exemption for Institutional Review Board according to Federal Statue 32 CRF Ch. 1(7-1-97 Edition) Section 219.101(b)(4).

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Dulan G, Rege RV, Hogg DC, et al. Content and face validity of a comprehensive robotic skills training program for general surgery, urology, and gynecology. Am J Surg 2012;203:535-9. [Crossref] [PubMed]
2. Dulan G, Rege RV, Hogg DC, et al. Proficiency‐based training for robotic surgery: construct validity, workload, and expert levels for nine inanimate exercises. Surg Endosc 2012;26:1516-21. [Crossref] [PubMed]
3. Instructions for Use (IFU). Available online: https://www.fda.gov/media/128446/download, accessed 15 December 2019
4. Satava RM, Stefanidis D, Levy J, et al. Proving the Effectiveness of the Fundamentals of Robotic Surgery (FRS) Skills Curriculum: A Single-blinded, Multispecialty, Multi-institutional Randomized Control Trial. Ann Surg 2019;31: [Epub ahead of print]. [PubMed]
5. Smith R, Patel V, Satava R. Fundamentals of robotic surgery: a course of basic robotic surgery skills based upon a 14-society consensus template of outcomes measures and curriculum development. Int J Med Robot 2014;10:379-84. [Crossref] [PubMed]
6. Gallagher AG, O’Sullivan GC. Fundamentals of Surgical Simulation: Principles and Practice (Improving Medical Outcomes – Zero Tolerance), Springer Verla, 2012: Chapter 8.
7. Gallagher AG, Ritter EM, Champion H, et al. Virtual reality simulation for the operating room: proficiency-based training as a paradigm shift in surgical skills training. Ann Surg 2005;241:364-72. [Crossref] [PubMed]
8. Gallagher AG. Metric-based simulation training to proficiency in medical education: What it is and how to do it. Ulster Med J 2012;81:107-13. [PubMed]
9. Satava RM, Gallagher AG. Next generation of procedural skills curriculum development: Proficiency-based progression. J Health Specialties 2015;3:198-205. [Crossref]
10. Zevin B, Levy J, Satava RM, et al. A Consensus-Based Framework for Design, Validation, and Implementation of Simulation-Based Training Curricula in Surgery. J Am Coll Surg 2012;215:580-6.e3. [Crossref] [PubMed]
11. Collins J, Levy J, Stefanidis D, et al. Utilizing the DELPHI PROCESS to Develop a Proficiency-based Progression Train-the-trainer Course for Robotic Surgery Training Eur Urol 2019;75:775-85.
12. Didactic course ISE. Available online: www.frsurgery.org Accessed 8 January, 2020
13. Vetter MH, Green I, Martino M, et al. Incorporating resident/fellow training into a robotic surgery program. J Surg Oncol 2015;112:684-9. [Crossref] [PubMed]