Skip to main content

Advertisement

SpringerLink
Log in

Surgical Coaching for Operative Performance Enhancement (SCOPE): skill ratings and impact on surgeons’ practice

  • 2020 SAGES Oral
  • Published:
Surgical Endoscopy Aims and scope Submit manuscript

Abstract

Background

Evidence for surgical coaching has yet to demonstrate an impact on surgeons’ practice. We evaluated a surgical coaching program by analyzing quantitative and qualitative data on surgeons’ intraoperative performance.

Methods

In the 2018–2019 Surgical Coaching for Operative Performance Enhancement (SCOPE) program, 46 practicing surgeons in multiple specialties at four academic medical centers were recruited to complete three peer coaching sessions, each comprising preoperative goal-setting, intraoperative observation, and postoperative debriefing. Coach and coachee rated the coachee’s performance using modified Objective Structured Assessment of Technical Skills (OSATS, range 1–5) and Non-Technical Skills for Surgeons (NOTSS, range 4–16). We used generalized estimating equations to evaluate trends in skill ratings over time, adjusting for case difficulty, clinical experience, and coaching role. Upon program completion, we analyzed semi-structured interviews with individual participants regarding the perceived impact of coaching on their practice.

Results

Eleven of 23 coachees (48%) completed three coaching sessions, three (13%) completed two sessions, and six (26%) completed one session. Adjusted mean OSATS ratings did not vary over three coaching sessions (4.39 vs 4.52 vs 4.44, respectively; P = 0.655). Adjusted mean total NOTSS ratings also did not vary over three coaching sessions (15.05 vs 15.50 vs 15.08, respectively; P = 0.529). Regarding patient care, participants self-reported improved teamwork skills, communication skills, and awareness in and outside the operating room. Participants acknowledged the potential for coaching to improve burnout due to reduced intraoperative stress and enhanced peer support but also the potential to worsen burnout by adding to chronic work overload.

Conclusions

Surgeons reported high perceived impact of peer coaching on patient care and surgeon well-being, although changes in coachees’ technical and non-technical skills were not detected over three coaching sessions. While quantitative skill measurement warrants further study, longitudinal peer surgical coaching should be considered a meaningful strategy for surgeons’ professional development.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. American Board of Medical Specialties (2019) Continuing board certification: vision for the future commission—final report. Chicago, IL

  2. Colenda CC, Scanlon WJ, Hawkins RE (2019) Vision for the future of continuing board certification. JAMA 321:2279–2280. https://doi.org/10.1001/jama.2019.4815

    Article  PubMed  Google Scholar 

  3. Jaffe TA, Hasday SJ, Knol M, Pradarelli J, Pavuluri Quamme SR, Greenberg CC, Dimick JB (2018) Strategies for new skill acquisition by practicing surgeons. J Surg Educ 75:928–934. https://doi.org/10.1016/j.jsurg.2017.09.016

    Article  PubMed  Google Scholar 

  4. Greenberg CC, Ghousseini HN, Quamme SRP, Beasley HL, Wiegmann DA (2015) Surgical coaching for individual performance improvement. Ann Surg 261:32–34

    Article  Google Scholar 

  5. Pradarelli JC, Yule S, Smink DS (2020) Performance coaching for practicing surgeons: enhancing clinical performance, well-being, and trainee experience. J Surg Educ. https://doi.org/10.1016/j.jsurg.2020.01.014

    Article  PubMed  Google Scholar 

  6. Pradarelli JC, Delisle M, Briggs A, Smink DS, Yule S (2019) Identifying naturalistic coaching behavior among practicing surgeons in the operating room. Ann Surg. https://doi.org/10.1097/SLA.0000000000003368(Epub ahead of print)

    Article  PubMed  Google Scholar 

  7. Birkmeyer JD, Finks JF, O’Reilly A, Oerline M, Carlin AM, Nunn AR, Dimick J, Banerjee M, Birkmeyer NJO (2013) Surgical skill and complication rates after bariatric surgery. N Engl J Med 369:1434–1442

    Article  CAS  Google Scholar 

  8. Pradarelli JC, Havens JM, Smink DS (2019) Facilitating the safe diffusion of surgical innovations. Ann Surg 269:610–611. https://doi.org/10.1097/SLA.0000000000003134

    Article  PubMed  Google Scholar 

  9. Min H, Morales DR, Orgill D, Smink DS, Yule S (2015) Systematic review of coaching to enhance surgeons’ operative performance. Surgery (US) 158:1168–1191

    Article  Google Scholar 

  10. Valanci-Aroesty S, Alhassan N, Feldman LS, Landry T, Mastropietro V, Fiore J, Lee L, Fried GM, Mueller CL (2020) Implementation and effectiveness of coaching for surgeons in practice—a mixed studies systematic review. J Surg Educ. https://doi.org/10.1016/j.jsurg.2020.01.007

    Article  PubMed  Google Scholar 

  11. Kanters AE, Shubeck SP, Sandhu G, Greenberg CC, Dimick JB (2018) Justifying our decisions about surgical technique: evidence from coaching conversations. Surgery 164:561–565. https://doi.org/10.1016/j.surg.2018.04.033

    Article  PubMed  PubMed Central  Google Scholar 

  12. Greenberg CC, Ghousseini HN, Pavuluri Quamme SR, Beasley HL, Frasier LL, Brys NA, Dombrowski JC, Wiegmann DA (2018) A statewide surgical coaching program provides opportunity for continuous professional development. Ann Surg 267:868–873. https://doi.org/10.1097/SLA.0000000000002341

    Article  PubMed  Google Scholar 

  13. Shubeck SP, Kanters AE, Sandhu G, Greenberg CC, Dimick JB (2018) Dynamics within peer-to-peer surgical coaching relationships: early evidence from the Michigan Bariatric Surgical Collaborative. Surgery 164:185–188. https://doi.org/10.1016/j.surg.2018.03.009

    Article  PubMed  PubMed Central  Google Scholar 

  14. Stefanidis D, Anderson-Montoya B, Higgins RV, Pimentel ME, Rowland P, Scarborough MO, Higgins D (2016) Developing a coaching mechanism for practicing surgeons. Surgery 160:536–545. https://doi.org/10.1016/j.surg.2016.03.036

    Article  PubMed  Google Scholar 

  15. Palter VN, Beyfuss KA, Jokhio AR, Ryzynski A, Ashamalla S (2016) Peer coaching to teach faculty surgeons an advanced laparoscopic skill: a randomized controlled trial. Surgery (US) 160:1392–1399. https://doi.org/10.1016/j.surg.2016.04.032

    Article  Google Scholar 

  16. Mutabdzic D, Mylopoulos M, Murnaghan ML, Patel P, Zilbert N, Seemann N, Regehr G, Moulton CA (2015) Coaching surgeons: Is culture limiting our ability to improve? Ann Surg 262:213–216

    Article  Google Scholar 

  17. Greenberg CC, Klingensmith ME (2015) The continuum of coaching: opportunities for surgical improvement at all levels. Ann Surg 262:217–219

    Article  Google Scholar 

  18. Pradarelli JC, Yule S, Panda N, Craig M, Lowery KW, Lagoo J, Ashley SW, Gee DW, Waters PM, James TA, Knight J, Gawande AA, Smink DS (2020) Surgeons’ coaching techniques in the Surgical Coaching for Operative Performance Enhancement (SCOPE) program. Ann Surg (accepted)

  19. (2006) The non-technical skills for surgeons (NOTSS) system handbook v1.2: structuring observation, rating and feedback of surgeons’ behaviours in the operating theatre. University of Aberdeen, Aberdeen, Scotland

  20. Martin JA, Regehr G, Reznick R, Macrae H, Murnaghan J, Hutchison C, Brown M (1997) Objective structured assessment of technical skill (OSATS) for surgical residents. Br J Surg 84:273–278

    CAS  PubMed  Google Scholar 

  21. Yule S, Flin R, Maran N, Rowley D, Youngson G, Paterson-Brown S (2008) Surgeons’ non-technical skills in the operating room: reliability testing of the NOTSS behavior rating system. World J Surg 32:548–556

    Article  Google Scholar 

  22. Jung JJ, Borkhoff CM, Jüni P, Grantcharov TP (2018) Non-technical skills for surgeons (NOTSS): critical appraisal of its measurement properties. Am J Surg 216:990–997

    Article  Google Scholar 

  23. Yule S, Parker SH, Wilkinson J, McKinley A, MacDonald J, Neill A, McAdam T (2015) Coaching non-technical skills improves surgical residents’ performance in a simulated operating room. J Surg Educ 72:1124–1130

    Article  Google Scholar 

  24. Lipsitz SR, Molenberghs G, Fitzmaurice GM, Ibrahim J (2000) GEE with Gaussian estimation of the correlations when data are incomplete. Biometrics 56:528–536

    Article  CAS  Google Scholar 

  25. Lipsitz SR, Fitzmaurice GM, Ibrahim JG, Gelber R, Lipshultz S (2002) Parameter estimation in longitudinal studies with outcome-dependent follow-up. Biometrics 58:621–630. https://doi.org/10.1111/j.0006-341X.2002.00621.x

    Article  PubMed  Google Scholar 

  26. Starks H, Trinidad SB (2007) Choose your method: a comparison of phenomenology, discourse analysis, and grounded theory. Qual Health Res 17:1372–1380. https://doi.org/10.3233/JAD-151150

    Article  PubMed  Google Scholar 

  27. Mazzocco K, Petitti DB, Fong KT, Bonacum D, Brookey J, Graham S, Lasky RE, Sexton JB, Thomas EJ (2009) Surgical team behaviors and patient outcomes. Am J Surg 197:678–685

    Article  Google Scholar 

  28. Cooper WO, Spain DA, Guillamondegui O, Kelz RR, Domenico HJ, Hopkins J, Sullivan P, Moore IN, Pichert JW, Catron TF, Webb LE, Dmochowski RR, Hickson GB (2019) Association of coworker reports about unprofessional behavior by surgeons with surgical complications in their patients. JAMA Surg. https://doi.org/10.1001/jamasurg.2019.1738(Epub ahead of print)

    Article  PubMed  PubMed Central  Google Scholar 

  29. Shanafelt TD, Balch CM, Bechamps G, Russell T, Dyrbye L, Satele D, Collicott P, Novotny PJ, Sloan J, Freischlag J (2010) Burnout and medical errors among American surgeons. Ann Surg 251:995–1000. https://doi.org/10.1097/SLA.0b013e3181bfdab3

    Article  PubMed  Google Scholar 

  30. Shanafelt TD, Oreskovich MR, Dyrbye LN, Satele DV, Hanks JB, Sloan J, Balch CM (2012) Avoiding burnout: the personal health habits and wellness practices of US surgeons. Ann Surg 255:625–633

    Article  Google Scholar 

  31. Bonrath EM, Dedy NJ, Gordon LE, Grantcharov TP (2015) Comprehensive surgical coaching enhances surgical skill in the operating room: a randomized controlled trial. Ann Surg 262:205–212. https://doi.org/10.1097/SLA.0000000000001214

    Article  PubMed  Google Scholar 

  32. Dyrbye LN, Shanafelt TD, Sinsky CA, Cipriano PF, Bhatt J, Ommaya A, West CP, Meyers D (2017) Burnout among health care professionals: a call to explore and address this underrecognized threat to safe, high-quality care. National Academy of Medicine. https://nam.edu/burnout-among-health-care-professionals-a-call-to-explore-and-address-this-underrecognized-threat-to-safe-high-quality-care/. Accessed 21 Apr 2019

  33. Jung JJ, Jüni P, Lebovic G, Grantcharov T (2020) First-year analysis of the operating room black box study. Ann Surg 271:122–127. https://doi.org/10.1097/SLA.0000000000002863

    Article  PubMed  Google Scholar 

  34. Greenberg CC, Dombrowski J, Dimick JB (2016) Video-based surgical coaching an emerging approach to performance improvement. JAMA Surg 151:282–283. https://doi.org/10.1001/jamasurg.2015.4442

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

The authors would like to acknowledge Ted James, MD, for his essential contribution as an institutional coaching champion for the SCOPE program at Beth Israel Deaconess Medical Center and Atul Gawande, MD, MPH, for his strategic guidance with surgical coaching at Ariadne Labs.

Funding

This work was supported by a grant from the Risk Management Foundation of the Harvard Medical Institutions, Inc., a part of CRICO.

Author information

Authors and Affiliations

  1. Ariadne Labs, Brigham and Women’s Hospital and the Harvard T.H. Chan School of Public Health, 401 Park Drive, 3rd Floor West, Boston, MA, 02215, USA

    Jason C. Pradarelli, Steven Yule, Stuart R. Lipsitz, Nikhil Panda, Kurt W. Lowery & Douglas S. Smink

  2. Department of Surgery, Brigham and Women’s Hospital, Boston, MA, USA

    Jason C. Pradarelli, Steven Yule, Molly Craig, Stanley W. Ashley & Douglas S. Smink

  3. Department of Surgery, Massachusetts General Hospital, Boston, MA, USA

    Nikhil Panda & Denise W. Gee

  4. Department of Orthopedic Surgery, Boston Children’s Hospital, Boston, MA, USA

    Peter M. Waters

  5. Kansas Coaching Project, Center for Research on Learning, University of Kansas, Lawrence, KS, USA

    Jim Knight

Authors
  1. Jason C. Pradarelli
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Steven Yule
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Stuart R. Lipsitz
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Nikhil Panda
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Molly Craig
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Kurt W. Lowery
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Stanley W. Ashley
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Denise W. Gee
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Peter M. Waters
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Jim Knight
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Douglas S. Smink
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jason C. Pradarelli.

Ethics declarations

Disclosures

Dr. Pradarelli reports a research grant from Johnson & Johnson Medical Devices and Diagnostics Global Services, outside the submitted work. Dr. Yule reports personal fees from Johnson & Johnson Institute, outside the submitted work. Dr. Panda reports a contract agreement with Aptima, a human-centered engineering and performance assessment contractor of the DARPA/Department of Defense, outside the submitted work. Dr. Gee reports consulting fees from Medtronic and Covidien and being on the advisory board for New View Surgical, Inc., outside the submitted work. Dr. Knight is founder of Instructional Coaching Group, a company for training professional coaches to help teachers improve their instructional practices. Dr. Lipsitz, Ms. Craig, Mr. Lowery, Dr. Ashley, Dr. Waters, and Dr. Smink have no conflicts of interest or financial ties to disclose.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Elements of the qualitative analysis of surgeons’ exit interviews were accepted for oral presentation at the SAGES 2020 Annual Meeting (Cleveland, OH; August 2020).

Electronic supplementary material

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pradarelli, J.C., Yule, S., Lipsitz, S.R. et al. Surgical Coaching for Operative Performance Enhancement (SCOPE): skill ratings and impact on surgeons’ practice. Surg Endosc 35, 3829–3839 (2021). https://doi.org/10.1007/s00464-020-07776-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00464-020-07776-1

Keywords

Search

Navigation